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ABSTRACT « 

An evalMatioii model was developed to measure the 
effectiveness of pilot programs in vocational Education for 
handicapped persons in selected State schools and State hospitals in 
Texas. -The model was field-tested' by conducting evaluations of 16 
pilot programs in ^vocational education for the handicapped located in^ 
seven State schools/hospi^^als. Data were collected by evaluation 
teams from the field of vocational education or special education for 
the mentally handicapped. The testing indicated that the model was 
useful for collecHng and evaluating appropriate dat^ to determine, 
program effectiveness. It was concluded that the model program 
concepts were more applicable to vocational education programs in 
public secondary and postsecondary institutions than to pilot 
programs in the, state schools/hospitals. Model development, test 
procedures, and finclings are detailed in the report.Jvocational 
program evaluations are appended,^ including a 39- page section 
presenting the instruments used to obtain evaluatiofi statement 
ratings (questionnaires to conduct personal int«aS(4ews with 
directors, counselors, program instructors, vocational students, and 
employers). Thirty-one pages of computer program information and 
tabulated data conclude the document. (MF) o 
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ABSTRACT • 1 -^^^ " 

/ 

/ 

Purpose 

The purpose pf this study was to develop an evaluation model which 
could be utilized to determine the effectiveness of pilot programs in 
vocational education for the handicapped in terms of meeting established 
objectives of vocational education. This purpose was to be accom- 
plished by testing the model in an effort to obtain relative measures 
of selected state schools' and state hospitals' sucdess in accomplish- . 
ing the established objectives of vocational education. 

The Evaluation Model ^ 

Following.- an extensive review of the literature relative to educa- 
tional evaluation, six postulates were formed "concerning the expecta- 
tions of an evaluation model. These postulates were utilized to form 
the theoretical base for theji evaluation model arid related processes 
developed in this study, A description of the model's primary compo- 
nentSy related processes, and selected operational procedures- follows: 
1. ^.OBJECTIVES:' Four objectives were utilizedf which were univer- 
sal in nature and expressed desired outcomes for vocational 
education in terms of national interests. The degree of 
success experienced at an institution in accomplishing each 
objective was depicted by an objective achievement score* 
Each objective achievement score was calculated by sunnning 
the degree of goal fulfillment for all goals relative to a 
particular objective. 



2. GOALS: Numerous statements were utilized to interpret the 
vocational education objectives in terms of. desirable pro- 

V 

grammatic concepts roJative to the educational setting and 

particulaT programs included in the evaluation. The propor- 

^ »- 

tionate amount each goal contributed to the objective 
achievement score was determined by a weight factor/. Weight 
factors were derived from importance and effort r^atings 
assigned each goal statement by the individuals who were 
administratively responsible for the conduct of the programs 
evaluated • The degree to which each goal was fulfilled was 
dependent upon the sum of the ratings assigned the outcomes 
pertaining to the specific goal, 

3. OUTCOMES: The data base consisted of numerous process and 
^ product type data elemonts; process data determined the 

^ existence of desirable program features and- product data 
assessed the effectiveness of the features; Each data 
element was in the form of a personal interview question or 
a prescribed observation — both required a "Yes" or "No" 
answer. Each answer received a scaled rating based upon the 
* evidence available to support the respqnse. 

-'4. ''related PROCESSES: Data were collectefl via on-campusxyisits 
by an external evaluation team utilizing prepared data 
collection instruments. Data analysis procedures were 
developed which included computer programa utilized in datar- 
mining weight factors and objective achievaaattt acoraa. 
A pictorial and descriptive summary of the major components included 
in the evaluation model are presented on the following page. 
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tour objectives arc utilized which ariT'univursai in naCuro and express desired outcomes 
tor votatxonal education in terms or national interests* 

ihc dcgccc oi success experienced at .la institution in aocompHi>hing each objectivd is 
depicted by an obje.ctive achievement .scoto. 

Cacu objcuCive achiuvcQont ^corc ib ciUviiati'vl by rtumning ttu- degree of go.il fulfil l- 
J^^t. LK J&SJ^} % g^elat ive i j a ^ r t icu I xr v>ri j ec^lye . ^ 

"jnTacTduJ^^jIatiaa":^^^ it»v vu*. al i»M»a 1 <^vlvKatlon objectives in 

'tt'Hob ot d^^^sJrable prograinmat it concepts u latlvt- tv> the wUtcatiVmal sett Ln.g and par- 
ticular progr«i.i(is» to^bc incl«idod in Ui'' ev.dn.it Urn. 

lilt |»iop«^rti«'natc auK>unt each k^mI vvill t^ntMi-utc u* \ hv »'bJe«tWv' achiev^t^ient ",i.v»r<' 
detenu i nod by a wolght factor < * *^ 

^olglit iuctoi^ are derived Irom imporUircc .tud et l v 1 1 i a ings , s<,lgiM»d <iach goal ,^taH.-> 
ift^'ut by iiic individuals wlxo are admtnlstrat IvcW r»'sp»»n .n»lo fur— tlie conduct ol the 
f jMo>tra*Ti«: l > be evaluated. 

• Uie degn-e to which each goal tb t iliiiivMl i*> dependent tipv>n the siun of th»' ratings 
.♦hsij^ned the out come r> pertaining to the specific >;oa1. 



iliv data ba^o <.onbij»ts of nunervAis process and product ty{>c data i Icnents— proctb,*. data 
CO doterwlno the existence of desirable program teatures and product data to assess the 
ef tectiveness of the features. 

hath data cleucnl jls in the fiTn of a personal interview quest Ion- or a prescribed ob- 
servation — both require ^ "^'t'l^" or "No" answer- 

bach answer receives ^ £*J;iil§, ha>sed u {)on the evidence a v a^i lab ic t o support the responce, 



0^ 



Dac^a Is to be collected via cn-c^mpus visits by an evaluatil)n team utilizing prepared 
data collection instruments,^, 

Data ajuilvoi^ ,j».utedures havc'been developed which Include conipnter programs to utilize 
in delermini r >g weight t ac tors and objective achie yement score s. 



Ihe nodel was developed by proceeding in a systetrfhtic mannt:r from the objectives to the 
outcomes; a program is evaluated by pcoceedlng ffom the outcom es t o the objectives. 
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Field Testing 



The model and its related processes were field tested by condncfc- 

' . . <. ^ ^ . 

xng evaluations of 16 pilot program^s in vocational education for the* 

\ 

handicapped located in seven state schools/state hospitals in Texas^ 
Data were collected via on-campus visits by four member evaluitlon 
teams consisting of individuals who were knowledgeable in the field of 
vocational education or in the field of special education for the 
mentally retarded, * ** 

c ^ Evaluation Model Findings 

Through the field testing, it was found that: , " ^. 

!• Through use of the model, an abundance of factual decision- 
making information was provided for evaluative purposes^ 

2. The data base provided sufficient process and product data to 
determina program effectiveness and describe contributing 
factors. 

3. The data collection methods prescribed in the model were' both 
efficient and effective, 

4. The data analysis procedures provided for direct comparisons 
on either a program by urogram or institution by institution 
basis! ^ ^ 

5* The programmatic concepts expressed by the goals included in 
the model were more applicable to vocational education pro- 
grams in public secondary and post-secondary institutions than 
to pilot programs in the state' schools/state hospitals. 
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CHAPTER I »?' 

* INTRODUCTION " ' ^ ' 

In'an effort to expand the role of education in meeting the social 
and economic need? o£ society, billions of dollars, have been provided 
to educational agencies at all levels by many private and governmental 
agencies. Tangible returnrs for such large investme^nts have customarily 
been difficult to, identify. Thus, mnny of the funding agencies are now 
nskinn it their huge expenditures for education are producing the 
desired results." In effect, the funding*, agencies are askin^i'for, and 
even in somc^ cases requiring, evaluative information upon which to base 
policy, decisions^ These requirements are especially evident in feder^l^ 
assistance i5>rograms, e.g.. Title I and Title HI of Public Law 89-10 
(Elementary and Secondal-y Education Act, 1965) where the law €;xplicitly 
states. that fund recipients will maVe at least annual evaluation 
reports ^nd Public Law 90-576 (Vocational Education Amendments, 1968) 
where the law strongly impUcs that recipients make periodic program 
(»Yaluat ion. * 

Surh requirements and imp 1 i eatior^s for eva.luntlon seem reasonable. 

« 

Funding agencies and the public have the right to know whether their 

colossal* financial investments have resulted in increased benefits to 

the e^ducationpl consumer—the student. Of equal importance is th^fact 
' • * ' » 

•that educators themselves need evaluative information to provide 



Th'e citations on these pages follow the style of the Journal of 
Educational /Psychology. ^ ^ ^ ' . 
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rational bases for their decisions among aJtOTnati^^e l>lans and proce- 
duros. However, to justify requirements foY evaliiatjons does noc make 
them operational. Educators must respond to the implications of 
accountability and requirements for evaluation, and they must do so 
effectively. 

The Texas Education Agency, Division of Occupational Research 
and Development, expressed a desire to determine whether significant ^ 
changes relative to occupational success had been effected by allocat- 
ing^ special funds to plTot programs in vocat;aonal-.,educat.ion for the 



handicapped in state schools and state ho3pit^s in Toxas. Thus, it 



was timely that this study be conducted in an ef^rt to design a 
'vocational pragram-javaluation model that would, piovide decision makers 
with reliable information upon which to base rational decisions rela- 
tive to the effectiveness of a program in meeting the vocational 
education needs of the student clientele. 

Purpose and Objectives 

The purpose of this study was to develop "an evaluation model 
which could be utilized to determine the effectiveness of pilot .pro- 

grams in vocational . education *^or the handicapped in terms of meeting 

\ 

^ established objectives of vocational education. This purpose was to 
be accomplished by fulfilling two contributing objectives, namely: 
1. Xo test the model and*" the related processes ^necessary 
for evaluating, pilot: programs in vocational education 
for th'e handicapped-. 



2* To obtain a relative measure of selected state schools'/ 

state hospitals', success in accomplishing estaby.shed 

objectives of vocational education:, 

* « > 

Nature of the Problem 

A prime objective of the Vocational Education Amendments of 1968 
was to assist States in providing meaningful vocational education to 
individuals who, due to their handicapping conditi<^,. could not succeed 
JjCLJiLegular^ vocational education programs. To this end, the Vocational 
Bdacart±on^-'Amendmer\ts ^o.frr±9-6:8:XSec:. 122^ (c) (3^^ rrecfulre^ t ha t " ^t~7 
least 10 per centum of each State's allotment of funds appropriated 
under section, 102(a) for any fiscal year beginning after June 30, 1969, 
shall be used only for-- the^p,uxpj3oe^ set forth in paragraph 4(B) [voca- 
tional education for handicapped persons] of subsection (a)," 

The Vocational Edbcation Amendments of 1968 also provides the 
following definition for identifying handicapped individuals: ***The 
term 'handicapped*, when applied to persons, means persons who are 
mentally retarded, hafd of hearing, deaf, speech impaired, visually 
'handicapped, seriously emotionally disturbed, crippled or other health 
imp a i^ e J~ per sons who by~reason thereof^^requine ^p^ej::Xal_educ&.tion and 
ts:lnt&d^&ex:n^^ JiJbXX 

Because vocational edCication administrators in many states found 
difficult to interpret the true meaning of the term 'handicapped' 
as^ found in» the Vocational Education Amendments of 1968, a more inclu- 
sive definition was given in the Federal Register (1970) and is stated 

{ ^ 

as : 

. "15' 
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(o) "llanclic'iripped persons'' means mentally retarded, hard of 
he«irLnft, deaf, speech impaired, visually handirnpped, serioiKslv 
emotionally disLurhed, crippled, or other health impai red i)er- 
Kons who I)/ reason of their hi ndieappin)* condit ion cannot 
succeed in a vocational or eorsumer and homemakinj; education, 
program designed for persons without such handicaps, and who 
for th^t reason require special educational assistance or a ' 
modified vocational or consumer and homemaking education pro--* - 
» gram. (p. 7335) 

In response to the directives set forth in the Vocational Educa- 
tion Amendments of 1968, a committee composed of representatives from 
the Texas Rehabilitation Commission, the Division of Special Education 
cid Special Schools, and the Division of Occupational Education and 
-^^Imo-lo^-'began to develop* a specific plan for implementing vocational 



educatri-on--pr-og'rams-for-^the-Jiandicap.p„e^ i n the State of Texas- The pinn 



developed by this' committee was adopted by the State Board of Educatic/n 

on June 7, 1969 (Meyer, 1972), and called for state funded .pilot vp<r^- 

tional education programs for the handicapped to take the following 

three-phase approach: 

Phase 1 Exploratory, prevocation^l activities to include 
vocational assessment techniques leading to an 
* , occupational diagnosis. 

<^ 

Phase II Vocati.onal education programs in a shop or labora- 
tory setting either especially designed for the . 
? handicapped or in a regular vocational program 

with modified curriculum. 

Phase III Job placement, evaluation, and former student ^ , 
follow-up. (p. 9) 



Following the State Board of Education's action^ 10 state schools/ 
state hospitals received approval to conduct pilot^ vocational education 
programs for the handicapped during the 1969-70 academic term. Two 
additional state schools/state hospitals received approval for the 
1970-71 term and three more were added to the approved list for the 
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1971-72 term bringing the. current total to 15^— Because the pilot pro- 
grams *being conducted at the Texas School Cor the Blind and the Texas 

> 

School for. the Deaf are special purpose programs, they were not con- 
sidered for this study; thus, m total of 13 state schools or .state hos- 
pitals offerlxig 31 different pilot programs in vocational education for 
the handicapped remained w^ich could be used to test the evaluation 
' model ► . 

Also during this time span, 1969-72, 102 other educational units 

teducatior\ service^ centers, junior colleges,- and^ independent school 

" ~ - 

districts) received approval to conduct pilot vocational education 
programs for the handicapped • As these pilot programs evolved, local 



school distfipir~ahd "jlini*or~^ programs- became- primariLyJhase II 

components of the plan while the education service centers have exclu- 
sively Phase I components. The state schools and state hospitals have 
predominately Phase II programs with a few Phase I components attached • 
Phase III is the responsibility of, the Texas Rehabilitation Commission, 
either through its cooperative programsr or through direct service by 

one of its counselors* 

■» 

The vocational education programs for the handicapped* helnf^ con- 
. ducted in the^ independent school districts 'have^^ since been transferred 
to operating status with state program responsibilities currently 



resting with the respective regular .progratn staff? in the Division of 
Public School Occupational Programs • State level administrative 



responsibilities for the programs in the sta'te schools and state ' 
hospitals have remained with the Division of Occupational Research and 
* Development • ' 
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Because the 197A-75 academic term represents the last year that 
the vocational* education programs for the handicapped being conducted 
in the state schools and state hospitals are to be funded as pilot 
programs, the Texas Education Agency had determined that selected 
programs should be evaluated in an effort to determine their effec- « 
liveness in preparing handicapped individuals for the world of work. 
Therefore, it was the intention of this study to develop and test a 
program evaluation inodel which could be followed in evaluating^ pilot 
vocational education programs for the handicapped in selected state 
schools and state hospitals in Texa^'. — 

A Theoretical Base 

~ — — — f « 

A trace of the historical developments in the evolution of educa- 
tional evaluation has revealed a considerable increase in the domain 
considered as legitimate for investigation by the evaluator. During 
ihe formative yeart> of public education^in this country until the^ 
mid-thirt Los , evaluation was used almost as a synonym for testing.- 
TesLing theories had been developed on'the assumption that the pur- 
poses of evaluation were to, measure individual differences in pupils 
and to describe groQps of students. Comparisons, when appropriate, 
were made b etwe en groups or between a group and an jestablished n orm . 

Erom^theJ.aiLtejtLJJiijLt^^ until jthe a dvent of th e post-Sputnik 

subject-matter curriculum projects, a basic pattern of evaluation 
activities began to emerge. Evaluation developments during this time 
sptm were characterized' by an increased concern for t h e__hi ft har - o r d e r 
cognitive and affective objectives and the consequent inappropriateness 

'^!8 



ut comparative ovaluations Lhrou^^h Lho vise oC groups or nonm«* Kvalua- 
tions were primarily concerned with determining the decree to which ^ 
students were obtaining behaviorally stated objectives; yet the objec- 
tives themselves were not considered appropriate for evaluation. 

Following the inception of the large, governmentally endowed 
curriculum projects of the early sixties and the "increased emphasis 
being placed on educational accountability, educational evaluation 
became a primary concern of many educators. Partially responsible 
for this increased concern, were the evaluation requirements incor- 



"p o rat e d in to Xederal„,d5SJL5„tauce, I je- gislatioa^ I t b e cam e- r eadily 

apparent that the evaluation practices of the past wire not capable 

of meeting the new and emerging demands for decision-making infc^rma- 

tion. Thus, a new philos'ophy of evaluation began to emerge and the 

previousJ,y disjointed practices of evaluators began to develop into 

a formal technology. ^ 

In the formation of the ne^ technology, it wag determ\ned that 

neither the traditional experimental design nor the management tools 

developed for government, business, or industry was capable of pro- 

vLding the necessary iitformatlon Cor choosing among decision- 
t • 

alternatives. Dfscisioh-making questions could only be answered 
educational institution. 'Thus, the need arose for evaluation models. 



designed for educational situations, which would provide the necessary 
data upon which judgements could be made concerning the worth of what 
was being taught and the efficiency of the methods employed. Further- 
more, evaluation is currently conceived as a means of judging the 



ultimate worth or value of educational endeavor. Therefore, the*, 
program comparisons issue becomes a central issue in the development 
of an evaluation model. 

Thus, based upon the writings of evaluation theorists and model 
developers, the following points of consideration were used to form 
the theoretical base for this study: 

1. AiT evaluation model must be capable of providing factual 
decision-making inlormation to a specific audience.. 

2. An evaluation model must provxde^^ jneans for fuJLly describing 
aii .ispects of an educational program. ^ 



3. TJhe evaluation model must be capable of producing reliable 
decision-making information without relying on an experi- 
mental design* 

4. Evaluation models utilized in education should be designed 
and developed for educational situations, , 

5. The data analysis procedures employee}, in the program evalua- 
tion model «houJd allow- for direct program comparisons. 

6. The performance indices employed to determine the effectiv,e- 
ness of a program^ should be composed of both process and 
product criteria. 



Definition of Terms 



The following definitions describe terms which i^ere used in 
conducting this study: 
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I, Px'OQViWi b\oaluation Model — an example of a plan of proce- 

~ dure wh±(ih_i nav be linLtate(L _for determinlnp. the eff-ective- 

uess ol instruction, 
2* Vocational Education Objective -7 statements whfdh define the 
programmatic thrust and reflect the desired outcomes for > 
• vocational education in terms of state and community inter- 
ests and student benefits (Starr & Dief fenderfer , 1972) • 
3^ Goal Statement — a statement which provides a basis for 
producing.. a core of information and whose achievement con- 
stitutes a proportional fulfillment of a specific vocational 
~€duGat-lon^ob^eatilvjeu__. / 



Delimitations 

The* following delimitations -were applied to this study: 

X* Nineteen pilot programs in vocational education for the 

handicapped, oTfere4_by seven* state schools/state hospitals 
in Texas during the 1974-75 academics-term, were utilized to 
test the program evaluation model developed in this stiidyv-^^. 
2\ Only those students who were enrolled in a specific pilot 
program for a period of not lesa than three months were con- 
sidered-^s^.prospects far personal int^rvdews^ ^ „^ 



3, Only those employers who had a former student working for 
them' for a period of at least three months were considered 
Cor personal interviews • 



021 



' . Limitations 

This study was subject to the following limitations: 

1. It was recognized that the director. of education at~earch"^'f ~ 
the institutions selected may have given higher ratings to 
those goal statements which more nearly depicted the opera- 
tional procedures af'the pilot programs in his school or 
hospital, • , . ' 

2. It was recognized that the various members of the evaluation 
team perceived the operational procedures of the pilot pro- 

^ grams differently. 

~~Ba5Tc— Assump^t-ions 1 

This study was based on the following assumptions: 
1. The goal statements selected by personnel iu the Division of 
Occupational Research^and Development, Texas Education 
Agency, did in fact depict the pperational and procedural 
intents of the pilot programs in vocational education for 
the handicapped, 

2* The ^valuation team members' ratings of the qualified "Yes" 
and "No"^ answers were consistent among the various institu- , 
tions and programs included in the testing of the evaluation 
model. 



CHAETERJLL ' 

MOni'.l. nKVl'.LOl'MKNI- AND 'l"KS'M'KOCKI)llUI'S 

Modell? are g^TTera4rl7-^«vuvtu^ purpose of assisting the 

'thought process. Depending upon the precise role' the model is to play 
and the state o-f knowledge in the field, the model itself may range 
from a full-scale or miniature replica ♦•o. a completely abstract 
calculus. This kind of activity is valuable if it does not become an 
end in itself; there must be an effort to test the validity of the 
exercise (Taylor & Cowley, 1972). Thus, based on the theoretical ~ 
points of consideration cited in the previous chapter, this chapter 
will present a model for evaluating vocational education programs 

and describe the procedures that^wS'e~~f olToweaTrT to test 

the validity of the concept. 
* ♦ 

» Model Development ^ ^ ^ - 

The purpose of this study was to develop an evaluation model 
which could^ be utilized to determine the effectiveness of selected 
pilot programs in vocational education for the handicapped in state 
schools and state hospitals in Texas in terms of meeting established 
objectives of vocational education.. This purpose was to be accom- 
plished by fulfilling the two ^objectives of this study, namely: ^ 

1. To test the model and the related processes i\ecessary for ' 
evaluating pilot programs in vocational education for th^ 
handicapped. 
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2. To obtain a relative measure of selected state schools'/ 
state hospitals' success in accomplishing established 
objectives of vocational education. 
Because The Center for Vocational and Technical Education at The 
Ohio State University^ (hereafter referred to as The Center) had 
developed a -ayste.nLXor state evaluation of vocational education pro- 
grams (Starr & Dief f enderf er, 1972) whiclTaisTlpp^are^^ based 
on a number of the points of consideration deve±Dped in Chapter II, 
The Center system was utilized as a point of departure for the evalua- 
tion model developed in this study. However, the evaluation procedures 

developed by The Center were designed to be implemented in an educa- 

/ 

tional setting that differed substantially from the state ♦schools and 
state hospitals in Texas. Thus, apart from a' similar skeletal outline 
bAsic evaluation strategies, the processes and procedures developed 
in this model are germane .to the p^IplT progr^s in^vacati^nal^education 
for the handicapped. . , ^ T 

•In addition to being based on current concepts .of evaluation, the 
evaluation system developed by The Center also included other features 
which made^.t,a logical point of depaftu?e for the evaluation model 
developed in-*this^ study. .These desirable features included:. 

i 

1. High level ot involvement •from various* agencies, groups, 
and individuals in the formation and testing of the system. 
Involved individuals Included evaluation consultants and 
specialists, state directors* of vocational educatiort, voca- 
tional program specialists, psychologists, and sociologists. 



2. Use of two advisojiy;>committees and a third committee composed 
of Center specia^sts who reviewed the project plans, evalua- 
tion materials, a'nd procedures. 
_ 3* Field testing of the evaluation sys'tem in three states to 
determine the system's efficacy and efficiency* 

Although the model developed in this study utilized a number of 

■» • 

the features and processes developed by The Center, it differed in 
several respects and provided ^tensions in other areas* The primary 
differences and extensions were: , ^ 

1* The model developed in this project was utilized to evaluate 

pilot vocational education programs for the handicapped in 

* 

state schools and state hospitals; whereas The Center model 
was designed primarily for evaluating programs in public ' 
secondary and post-secondary Institutions* 
2. In this model, most of the .data collection instruments are * 
completed hy visitation team members; whereas in The Center 
model, ^tKepnajor^^ty^of tjie data collection instruments are 

completed by personnel at the institutiotrVeimg^ev^aluated, 

^ • '--^ ^^^^^^ 

3; The model utilized in this project provides ^a means for 

those individuals who are responsible for the conduct of the 
programs being evaluated to assign various weight factors to 
each of the goal statements^ The Center model does, not 
provide a means for assigning weights to goal statements. 

vA * IP ' 

4. The data analysis method utilized *ln this model provides 
for a direct comparison of the rate of achievement between 
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institutions. The Center model does not provide for instilti-. 

tion or program comparisons. 
Similar to the evaluation modol created by The Center, rho wkkW] 
developed in this study has as its main compon^ts objectives, goals,*- 
and outcomes. Figure 1 schematically depicts the relationship of the 
three components. 



Goal, 



Outcomes 



Objective 



Goal, 



Outcomes 



n 



Goal 



Outcomes 
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Figure 1. - Schematic diagram depicting the relationship of the stated 
objective to the goal statements and outcomes. * . 



In this model, the objectives reflect theji.desired outcomes ,forY' 
vocational educati/on in terms oE national interests. However, these 
objectives are insufficiently precise to be measured directly. 



2m 



Therefore, It* was necessary* to interpret each objective in the form ol 
numerous goal statomonts which arc sufficiently procLso to be measur- 
able., The oxtcnt to which' eiuh goal statement Is fulfUlcd is 
^i^ermined by- the number pf affirmative responses (outcomes) relativo 



rto each goal, Thus, ky first summing the number ofj[ affirmative out-« 

comes related to each goal statement and next, by summing the degree 

to which each goal statement was fulfilled,. -it becomes possible to 

obtdln a relative measure of an institution's success in accomplishing 

a particular objective, , • ' 

Two additional factors entering into this additive model are* the 

relative weights .assigned to each goal and the qualified rating given 

« 

to each outcome. The goal weights are derived from ratings assigned 

each goal statement by thos<i individuals- who. aro responsible for the 

conduct* o,f ,thc programs being evaluated. On a 5-point Likert-typo* 

scale, these^ individuals rate the'rclative importance of eath- goal 

statement to their respective urograms and the dcgroj of effort they 

would have to expend to incorporate the expressed concept into their 

•programs. Relative weights are thfen 'assigned to each godl statement 

■ * * I ^ « 

act:ord,i4i^ to the calculations depi\ted in Table, 1. 

Kach outcome in this model is diTectly related to a specLCic goal 

slVitement. Just as there are numerous f;onl,s lor cMch objective, there 

also\nre niimerpus outcomes for each goal. The outcomes, which cons'Li- 

t-Hj-t' the data base, are "Yes^" and "tNo" answers Co specific i»Cervi!ew 

quest [.ons and prescribed ohservations. Because It Ls extremely diffi- 

, ^\ • ' ^ 

** ^ • • ' * 

cult ^to answo;; most of the questions wll:h an unequivocal "Yes" or 

> * * 
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"No/'' provisions were made to qualify each answer^. Consequently, each 



• 


answer (outcome) is rated according to the 


following scale: 














{ 


' ' 3 = Yes, 


but evidence to suppotttthe 


answer is lacking* 

0 




^ * t 


7 = Yes, 


but evidente used to support 


> 

: the answer is questionable. 




. iO Yes, 


there .is strong evidence to 


support the answer. 








^ Table 1 , 
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Goal Weight Calculations for Impgrtance and Effort Ratings 
-J. Assigned to Specific Goal Statements' 
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^Once the goal weights are caloulaLed and the data aro collected, 
a relative measure of an Institution's suocess In aocomplishiiig a 
stated objertive can then be^^btaLned by subsLltuLing the obtained 
quantities into the following formula: 



2(w^s^) 

P = — =: X 100 where: P = percent of objective 

^1 * achievemenrt , 

w^ = weight factor assigned to 
each goal statement, and 

<* 

s^ = average accomplishment of 
' ^ each goal statement, i,o., 

number of questions per goal 
divided by ' the sum of^ the 
rating given those questions. 



Cuce an institution's percent of objective achievement is 
obtained for each program being evaluated, comparisons can be made 
either on a program--by-program basis or on an institution-by- ^ 
institution basis. If further analysis is required, comparisons 
could also be made nn a goal-by.-goal basis. 

This model, then, satisfies three of the six points of considern- 
tion listed' in Chapter I, Specifically, these points of consideration 
are: ■ - ^ ' ' ^ ' o 

Point of Consideration 3:^ TKe evaluation model must be capable 
of producing reliable decision-making information without relying on 
an experimental design. 

Point of Consideration 4: Evaluation models utilized in educa- 
tion should be designed and .developed, for educational situations. 
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^Point of Consideration 5: The data analysis procedures employed^ 
in the program evaluation model should allow for direct program 
comparisons* 

o 

* Vocational Education Objectives 

The vocational education objectives utilized in this study were 
formulated by Th e Ce nter fnr V nmM'nn^i anH Technical Education^, The 



Ohio State University. These objectives represent :the work of 

The Center personnel; evaluation .specialists, and advisory committees 

"I 

with final- validation by a natloral survey of state directors and 
head state supervisors of vocational education. Because these objec- 
tives represent the concepts of leaders in the field, of vocational 
education and because they are also sufficiently global to apply to 
any program at any level, these objectives, were" utilized as measurer 
ment indices in this study as .their degree of fulfillment provides an 
assessment of an institution's success in meeting the vocational 
education needs of its student clientele. 'The objectives are stated' 
as follows (Starr & Dief fenderfer , 1972): 

1. To provide vocational education to youth and adults who 
will be entering the labor force and to those who seek 
to upgrade their occupational competencies or learn new 
skills, 

2, To provide comprehensive vocational education which relate.9 
general and vocational education offerings to the voca- ~ 
tional objectives of students. 

3* To make vocational education increasingly accessible to 
those who desire it • ' — 

A, To provide quality vocational education which meets the 
' ^ vocational aspirations of people while being compatible 
with employment opportunities, (p, lA) 
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(Joal Statements 



The numerous functions tho goal staterocntB wore ral.led upon I o 
fulfill in this evaluation model made their final selection an arduous 
task. The goal statements were designed to: , • 

i; Produce a core of Information which would yield an assess- 
ment of the extent to which the established objectives of 
vocational education were being achieved • 
* 1. Reflect the vo.cational education intents of each instit,ution. 

as stated in their initial ^proposal to the" funding agency. 
3. Reflect the program decision-requirements of the funding 
agency, ^ * ' . 

. 4* Provide a basis upon which to make a .full description of 
\ the intents and activities of each program. 

Following a review of the goal statements included in the evalua- 
cion model developed by The Center and an indeptfi study of the propo- 
' &als submitted to the Texas' Education Agency by the various state 
schools/state hospitals for the purpose of establishing pilot programs 



vo 



goal 
the g 



cational education for the handicapped , a preliminary list of 
statements was formulated for this evaluation. To determine if 
:>al statements would satisfy the program decisidn-requirements of 



mten 



the f inding agency and lead' to a full description of each program's 



ii;s and activities, the preliminary list was submitted to personnel 
in> thi Division of Occupational Research and Development, . Texas 
Educatjion Agency, for their review and assessment. The final set of 



goal s 



tatem^nts then resulted in the following: 
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Goal statements relative t <v, Object I vp' 1 : 

V > 

if ' ^ 

1:01 An institution otlcring vocal' lonal education shunUl possess 
a statement of philosophy which reflects the institution's 
purpose and objectives relative to meeting the needs of the* 
student jplientele. > 

1:02 The institution should possess a financial plan which is 

adequate tcr assure the quality and continuity of the voca- 

tional education programs. * | , 

1:03^ The institution should possess formalized future plans for ' 
vocational education. 

1:0A The institution should utilize a' general adviso'ry committee 
^ in planning for vocational education and various occupa- 
tional advisory committees to assist in developing current 
curriculums for each program. , - ■ • 

Goal statements relative to Objective 2: 

2:01 General education personnel, professional personnel, and 

^ * ^ ' - ' — 

vocational^education personnel, should work together to 

organize their offerings and services in relation to the 

vocational education program. 
2:02 Specific program instruction should be based on written 

performance objectives which were derived from an analysis 

of required occupational competencies that need to be 

taught for a specific occupation. 
2:03 There should be a continual analysis and subsequent 

instructor updating of the occupational competencies that 

need to be taught for a specific occupation. 



2:04 Consumer education should be included as an integral part 

of the total vocational education program, 
2:05 Employer-employee relations should be taught a^ an integral 

part o£ the total vocational education program. 
2:06 Students enrolled in an effective vocational program shojuld 

^ show evidence of change and growth in attitudes. 
Goal statements relative to Objective 3: 

3:01 Students with reconcilable educational deficiencies should 
be encouraged to enroll in the vocational programs. 

3:02 Specialized remedial instruction should be made available 
to stud.ents who have educational deficiencies which act as 
serious barriers to successful program completion or job 
*^ placement, . ' 

3:03 The^potential„number of students the vocational programs 
can successfully accommodate ,should be enrolled. 

3:04 All vo.catJonal program? within the institution should be 



oPftM-ed on a twelve-month basis, 
3:05 The LnstituLipn should offer a structured career guidance 

program which enrolls all vocational students. 
Goal st^atemcnts relative to Objective 4: 

~4:"t)l The insTructioaO~pracjtices~ carried on in 'the "classroom "and 
laboratory should provide students with simulated work 
experiences reflective of what would be expected of them 

, i 

in a wage-earning situation* 
4:02 Major equipment and machine acquisitions should be com- 
mensurate with the program objectives^ and whan installed. 



the Items should be complete with nil snfety devices ;nul 
pjaced an i\ prevent ivo maintenance and. replnceinoiU 
schedule, * , * 

4:03 The vocational programs should be housed in adequate 

facilities that are well managed and free of health, fire, 
«(nd safety hazards • 

4:04 The institution should possess a formalized procedure for 

■ . . * , i 

placing both graduates and non-graduates in an employment- 

situation that is in or directly related to the area of 

their preparation, 
4:05 Students in the vocational programs should be involved in 

cooperative or other out~of-class work experiences • 
4:06 The vocational instructional staff members should be 

certified in -the areas in which they teach and be evaluated 

in terms of their teaching proficiency, 
4:07 Vocational instructors should use effective teaching 

methods, procedures, and instructional materials in the 

conduct of the teaching-learning process, 
4:08 The institution. should possess a formalized follow-up ' 

procedure which yields information relative to the ade- 
, quacy, V appropriateness, and effectiveness of the vocational 

education programs, 
4:09 An institution offering vocational education should conduct 

ah annual self-evaluation for the purpose of identifying 

Strong and weak points as a basis fpr upgrading the 

programs. ' - 



The numbering system applied to the above stated goals was 
devised to indicate the relationship pf the goal to an objective and 
to identify the series order of the goal statement. The numeral ! 
preceding the colon specifies the objective number while the numerals 
succeeding the colon identify the series order number of the goal 
statement. Each goal statement . retained this identification number 

r 

throughout the duration of the study. ^ 

' Underlying the phrasing of each ^oal statement were the assump-* 

"^tions that many educational programs are implemented without adequate 

planning and forethought; therefore, evaluation should be a process 

» which contributes, to further development and stabilization as well as 

assessment of a program. Furthermore, as Stevenson (1973) notes: 

• Those charged with responsibility for selecting evaluative 
criteria and those conducting evaluation? must keep' in mind 
the fact that programs will, change in'the^ vdirectibn of the 
stated critjaria. If those conducting .programs are aware of 
the criteria to be used in evaluation and if they believ^ 
that administration is sincere- in its designation of the items, 
^. then programs^will change toward these specified criteria. If 
J a climate of mutual trust arid concern can be created between 
administrators, ^valuators, and teachers, program improvement 
can result. SupeF^visory personnel can find in individual pro- 
gram evaluations some very specific suggestions for program 
supervision and improvement. (p. 77) " ^ 

- 'l ^ 

For these reasons, each goal statement was phrased in a positive 
nature and included the verb **should" .to indicate desired outccmes. 

Therefore, through the meticulous selectJLon and the veracious 
phrasing of each goal statement, two additional points of considera- 
tion are satisfied in the evaluation model. Specif ic£llly, these two 



points of consideration are: 
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' Point of. Consideration 1: An evaluation model must .b^ capable of 

providing factual decision-making information to a specif ic ^dience. 

Point of Consideration 2: An evaluation model must provide ^^ 

means for fuj-ly describing all aspects of an educational program. • 

t 

Data Base .Devej\opment 



In this model, dat^ requirements were sy stematically derived from 

program 'objectives and goal statements. The ptocedure utilized to 

♦ 

derive the data elementij in a systematic mann<ir consisted of (a) list- 
ing each goal statement, (b) listing under each goal statement the 
data elements required am order to determine if the goals were being 
achieved, and (c) listing under each data element the source of the 

< '.K ■ . \ ^ 

required intormation. This procedure led to the identification of si^ 
sources of information, namely, institution administrators, program 
instru&tors, vocational counselors, current students, former students' 
employers, and evaluation team members. After considering the data 
elements derived through this procedure and the organizational'^ struc- 
ture of the state schools/state hospitals, it was decided that the 
institutional administrators could best be represented by the director 
of education at each institution/ This decision was based on the 
assumption that the director of education possessed a basic knowledge 
of administrative recjuirements and operating procedures of the voca- ' 
tionai education programs. At this point in the development of the ^ 
evaluation procedures, it was also decided thrat the most efficient 
method of acquiring the required infonpation from institutional 
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personnel, students, and i!xmployers was through personal interviews. 
The basic reason for utllixinK thc^^ personnl interview as ^^^nta col- • ^ 
lection pr'ocedure stenmied ffbm the possibility that auditory and .> 
visual clues such as respondents' incidental comments, faelaJ an'd > 
bodily expressions, and tone of voice could be used to assist the 
vlsitatidn team members in applying the appropriate rating'to the datp 
elements. Because the personal interviews werfe to be. conducted in the 
respondents* institutional setting, it was determined that the visita- | 
tion team members could also make the* prescribed observations at the 
same time. • « 

In the development of the data base (specifically step (b) above), * 
consideration was also directed toward selecting the minimum number of 
data elements required to yield n reliable assessment df the extent to 
which the program go^ Is were being achieved and, at the same time, 
provide sufficient information to describe in detail the meaningful 
activities associated with each program. These expectations of the 
data elements required that a dynamic mix of process and product cri- 
teria be obtained, a mix which would yi.eld quantitative indicators of 
each institution's success in meeting the vocational education needs * 
of its student clientele. It was postulated, that through efficient 
administrative practices, reliable guidance procedures, and effective 
instructional methods, the pV<^ram goals could be achieved and the 
vocational education heeds of tli^a stiftlents fulfilled. Thus, the data 

W, 

base for. this study was formed by quantifying the interviewee's 
responses to specific questions and by \ecording the visitation team 



members V acknowledgements of prescribed ob.servntlons . Tnl>Le 2 Illits- 

L **** 

imtes thq number ;md typo of djita elementsV (oiitcome.«) as80cli<tt>d wltli 
each goal statement and with enc4i Individual or group. 

Th^ mix of process .^nd product criteria utilixed in this model 

I ^ " ' . ' 

covared the follpwing general areas for each of the above mentioned 



groups ; 



11 Director of Education — questions and observations concerning 
. the" institution's philosophy and objectives of vocational 

I education for the handicapped, evidence of future plans for 

^ vocational education, financial mauagement^^^f^the special 
funds provided for the pilot programs, a^dministrative prac- 
o tlces relative to the conduct 'Of the pilot programs, evidence 
that program and instructional staff evaluations have 
occurred^ and evidence of administrative support for advisory 
committees, placement services, and follow-up procedures. 

2. Vocational Counselor — questions and observations indicating 
the presence and continuing use of vocational information 
services, personal data collection, counseling services, and 
the existence of a program for identifying, appraising, and 
providing individualized remedial instruction to students"' who 
possess special personal or social handicaps*, 

3^ Program Instructor — questions and observations concerning the 
efficiency and effectiveness of instruction, including the 
use of behavioral objectives, and individualized instruction; 
^ instructor preparation and recency of work experience; > ^ 



Goal 

. No. 



Table 2 ' . . , ' 

Number and Type of Data Elements Associated with each 
Goal Statement by Individual or Group . 



.Data 
Elonent 
Type 



Individual or Group 



"DOE^ 



INS' 



COU' 



Goal 
No. 
Total 



1:01 



Ques. 
Obs. 



10 



10 



1:02 



1:03 



Ques.' 
Obs. 



Ques. 
Obs. 



13 



4 1 

i 



1:04 



Ques. 
Obs. 



U 



2:01 



Ques. 
Obs. 



8. 



)bjective 1 Total « 44 
10 



2:02 



Ques. 
Obs. 



2:03 



Ques. 
pbs. 



2:04 



Ques*. 
Obs. 



7 .. 



2:05 



Ques. 
Obs. 



- 8 



12 



2:06 



Ques. 
Obs. 



« 10 



3:01 



Ques. 
Obs. 



Objecltive 2 Tot«V - 64 



s(|tive ^ 



3:02 



Ques. 
Obs. 



3:03 



Ques. 
0 Obs. 



10 



3:04 



Ques. 
Obs. 



3:05 



Ques. 
Obs. 



26 - 5 
1 



33 



Objective 3 Total - . 66 
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Table 2 - Continued 
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Goal 
No. 


Data 
Element 
Type 


V 


Individual or 


Group 


Goal 


DOE* ^ 




/ cou'^ 




No. 
Total 


4:01 . 


Ques. 

Obs. 


> 


••1' 

7 , 




5 


6 


7 


4:02 


Ques . 
Obs. 




"4 1 ■ 




, 9 ' 

1 1 . 

/' 


6 


4 


4:03 


Ques* 
Obs. 




4 

12 




■ 


' 4 


12 


4:04, 


' Ques. 
" Obs- 


5 


2 


1 


1 


9 


0 


4:05 


Ques . 
Obs. 




3 

1 




t 


3 


p 


4': 06 


Ques. ' 
UJt)S.^ 


7 


6 




■ 


13 


0 


4:07 


Ques. 
Obs. 




9 

5 


• 




9 


• 5 


4:08 


Ques. 
Obs. 


8 ' 


2 






10 


0 


4:09 


Ques. 
^ Obs.^ 


5 


3 






8 


0 












Objective 4 Total - 


• 96 . 


Quest icn TuLal 

' Observation Total 


- 11 . 


100 

43 


31 

1 


20 11 , 
a 0 


215 


55 


Total Ques. & Obs. 


64 


143 


32 " 


'20 Hi 
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*DOE • director, of education. 



'^INS • program Instructor. 

^C'OU - vocational counselor. 

^SXU « vocational student. 

^EMP - former st^udents' employers. 



instructor "contributions to budget planning, program planning, 
, selection and use of- advisory committees^ placement services, 
and' former student foJlow-up» 
, Vocational Student — generally product-type questiotti? tp i 

determine ±f the students were actually receiving efi'ective s 

instructio'h and reliable guidance sj^vices, 

• I 
5. Former Students' Employera — product-type questions relative^ 

to work skills and employer/employee relations* / * 

Former students, another source of valuable et^aluative data, were 

not inadverteht>ly omitted from this study. Due primarily to financial 

restraints, former students were' not included as' a- source of evaluative 

• • • 

d*ata. Because formec student's are few in number and dispersed through- 

out the entire state and bicause the Only method of obtaining reliable 

information from them would be through persdhal interviews ,> the finan*- 

cial costs '^involved in contacting theae individu&ls made such an 

endeavor impractical* However, two additional efficiency factors were 

considered in the development of the data base. Specifically,, thfese 

were: ' 

t 

1. The time frame required for collecting data. 

2. The personnel required to manage the data collections. 

• Thus, through, an indepth analysis of the information Requirements 

aniJ the systematic selection of the data elements, the final point of 

coaaldwration listed in the precedini* ctapter ma fulflllad, I.e., 

Point of Consideration 6: The performance indices employed to deter- 

. — % 
mine the effectiveness of a program should be com^sed of both process^ 

and 'product criteria.- 



' * . j^i^ Instrrument Davt?lopffient? " */ 

Because .the evaluation model lormulated Jnj this study differed 

substantially from previously developed models, the data requkements 

also differed which neo^ssitated the development of 'data collection 

A 

;instruments designed specif ically *f or ^this modelt^ To satisfy the 
infbrroatipn requirements . of this model, two types of data collection 
instruments were required. First, an instrument was required' to 
obtain the importance and effort rating for each goal statement, and 
secondly, a set , of instruments was required to* obtain the inter-: 
viewee's Response to specific questions and the interviewer's a know- 
ledgepent ojf prescribed observations. . ,^ g 

The instrumeni: developed to obtain the Importance and effort 
ratings was' designed to be completed by the Director of Education at^ 
each of the* institutions included in bhe evaluation study. .This . 
instrument, Evaluation Statement Ratings" (Appendix A), contained 
a listing of the previously described goals and two^5-point I,ikert-typ 
scales for .each goal statement. The two 5~point Likert-type scales 
. (one which ranged from little importance to great importance; the ' 

second which ranged from little effort to great ef^fort) were utilized 

/ 

to obtain the importance and effort ratings from/which subsequent 



•weight factors could be calculated and assignee^ to each goal state- 
ment. ^ ^ • 

Once the data base was formulated, ,as d^scrib(»d in the previous 

section^ ^nd the individual or grouTp who pgTssessed the desired 

/ 



information identified, developmonl of the scroml tvpe of instrumont 
beoarae a matter of changing the data cloment listings to ^juestions. 
This' set of instruments (Appendix B) consisted of the specific ques- 
tions to be posed to the identified groups and the prescribed - 
observations to be made by the visitation team'members* Each ipstru- . 
ment in this set was designed to be completed by a visitation team 
member as he/she interviewed a selected individual* Question^ and 
prescribed observations were grouped according to subject 'and 
arranged in such a manner as to make the interview resemble a*^' normal 
conversation. To assist the interviewer in distinguishing between 
questions and .prescribed observations, the questions to be asked^ the 
respondents were placed at the left-hand margin and began with a verb 
while the prescribed observations were indented, written in sentence 
form, and enclosed iii an c^iitline. As' a means of associating e^h data 
element with a goal statement for data processing purposes, goal 
numbers were placed after each question and prescribed observation. 

Decisions relative to the format as well as the formation and 
phrasing of the specific items eventually included in this set of 
instruments werfe influenced by previously developed evaluation pro- 
cedures. Primary consideration was accorded the following sources: 
A System for State Evvduation of Voaatioruzl Education (Starr & 
Dieffenderfer, 1972); Instruments and Procedures for the Evaluation 
of Vocational/Technical Education Institutions and Programs (Xmeric. 
Vocational Association, 1971); A Device for Evaluatiyig Departments 
of Vocational Agriculture in Arizona (University of Arizona, 1967) ; 
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Vocational Education Pvogvom Study {18022: Trainiuij Manual (Tobins. 
197A); Evaluative Critevia jov Vocational and Vcobhlral VvoijVimu\ 
(Reynolds, Grobman, & McGee, 1967); Sclf^evaluatio}i'' Guide for Loyal 
Districts for Vocational Education of Handicapped Students (Meyer, 
1972); Handbook for Self-evaluation of Programs and Services to the 
Disadvantaged and the flandicapped at Community Colleges Under the 
Vocational Education Amendments of 1968 (Tadlock Associates, 1972); and 
Evaluation for Enviromental Education (Ambry, 1972). 



Establishing Content Validity of. the 
Data Collection Instruments 



In an effort to .establish the gontent validity of the data collec- 
tion instruments, aside from the systematic procedure^ utilized to 
identify the data elements, the listings of the goal statements and 
corresponding data elements as well as a draft of^ the data collection 
instruments were submitted Xo a jury of experts for their review* The 
cbmpositioh of this jury , included : Texasr A&M University professors who 
had previously demonstrated competence in educational evaluation and 
who were familiar with this study; Texas Education Agency, Division of 

Occupational Research and Development, personnel who posses^^d state 

' ' ^' ' \ 

level administrative responsibilities for the pilot programs in^voca- 

tional education for the handicapped; Texas Education Agency, Division 

/ * 
of Special Education and Special Schools, personnel who wer^ familiiir 

with educational programs for the handica^ped"^; and Texas Department of 

Mental Health and Mental Retardation personnel who possessed state 



0 AAi 



33 



level administracivo responsibilities for the educational programs In 
the state schools/state hospitals. 



Following a dis^ussi^^ the : 

purposes and objectives of this study- as well as an explanation of 
the proposed data collection methods, this group was asked ^o review 
the prepared materials in an effort to identify data gaps and to deter- 
mine wljether^the information sought was actually available. Pertinent 
suggestions for additions, deletions, and changes were subsequently 
incorporated into the data collection instruments, f 



* ' Test Site Selection 

In the, selection of the state schools/state hospitals to be 
included in this study, primary consideration was given to obtaining 
a group of institutions that (a) were geographically representative 
of the state, (b) provided a cross-section of the pilot programs in 
vocational education for the handicapped, and (c) included both state 



schools and state hospitals. Upon consideration of these factors, 
personnel in the Division of Occupational Research and Development, . 
who were responsible for selecting the institutions to be included in 
this, study, identified seven state schools/state hospitals. In addi- 
tion, an eighth institution was identified for pilot testing purposes* 
The seven institutions identified for inclusion in this study offered 
a combined total of 17 pilot programs in vocational education for the 
handicapped while the Institution selected for pilot testing purposes 
offered two programs. ' 
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visitation Team Selection and Orle ntat ion/r^ — 

Very- often the term "evaluation" arouses an unprecedented level 
of anxiety in m-any individuals — particularly when the individuals in. 
charge of an educational endeavor are informed that their programs are 
about to be evaluated. In the design, and development of' this evalua- 
tion model, considerable attention was given to methods of lessening 
the amount of apprehension that would be experienced by those individ- 
uals conducting the educational programs to be evaluated. One methpd 
employed* to decrease, anxiety was to structure the interview format in 
such a jnanner that the ,questioning^,process would resemble a normal 
conversation. It was postulated, however, that the most effective . . 

y " ■"' 

method of reducing VoveralT." apprehension would be to collect the 
required evaluative information in the least amount of time. Thus, the^ 

evaluation team approach to the data collection process was utilized 

' " ' \ 

In this model. The generalization was? that, if several individuals ~ 

could simultaneously gather information, the data collection process 

would Be greatly expedited and the amount of time the institutional 

representatives would have to contribute to the evaluation process 

would be shortened considerably; thus, resulting in an "overall" 

decrease in the amount of anxiety generated by the evaluation. 

Another reason for utilizing an evaluation team was to include 

personnel in the evaluation process who were knowledgeable in the area 

of mental health/mental retardation and who were acquainted with the 

organizational structure and the operating procedures of the state 

0- 4§f. 



schools /state hospitals. After considering these factors and the 
amount of data to be collected at each institution, it w^s determined 
that a team composed of four individuals would be the most efficient 
in this setting. It was also determined' that the team would be 
most effective if it was representatjive of vocational education, ,the 
Texas Department of Mental Health and Mental Retardation, and the 
Texas Education Agency, Division of Occupational Research and Develop- 
ment and Division of Special Educatioo- and Special Schools. This 
researcher became the vo'^ational education rep;resentative while ,per- 
sonnel from the above mentioned agencies co^i|jle^ted the team member- 
ship*"* Due to the compressed schedule of the on-campus visits to. the 
various state schools/state hospitals, it was necessary to select two 
individuals from each of the^agencies to serve. on the evaluation 
team. 

JFol lowing, the selection of the visitation team members, an 
ur-ienjtaliun meeting was held in the offices of7~the~Dlyision of Occupa- 
tional Research and Development for the purpose of acquainting each 
individual with the evaluation procedures. . At this meeting, the 
purposes and objectives of the study were discussed and the operational 
procedures of the evaluation model were explained* Each data collec- 
tion instrument and each 'data element contained therein was reviewed 
for the purpose of removing any ambiguities and determining the degree 
of evidence ,that would be required to qualify each question and 
observation contained in the specific data collection instruments. 
The interviewing techniques- presented in the bboklet entitled 



Vocational Education Program S;tudii if 8022: Tminintj Manual^ which were 
utilized by a private research firm. Decision Making Informatlan, In 
conducting a national survey of vocational education programs Cor ,the 
handicapped, were adopted for this study and were discussed at "this 
time.» Special emphasis .was placed on" probing techniques (Jobias, 
1974) which are **. . . the methods of questioning used by interviewers 
to aid respondents* in clari'tying any answers that are incomplete, 
vague, ox ambiguous** (p. 9). 

• ' • - ■ 

^ <» 

Interviewing assignments were alsp made at this orientation 
meeting. To add reliability to th^ study, it was determined that 
individuals possessing similar responsibilities at each of " the state 
schools/state hospitals would be interviewed by the same visitation 
team member. Thus, after also considering the logistics^ of this 
evalCiation model and the expertise possessed by the various team 
members, the following interviewing assignments were made: 

3rr~preX3[s~Deparrffientn3^ 7 
personnel would interview the Director of Education at each 
^ institution. 
,2. Texas Education Agency, Ditrlslon o^ Special Education and 

Special Schools, personnel would Interview students. 
.3. Texas Education Agency, Division of Occupational Research, 
and Development, personnel would interview program 
^ instructors. 
A. This writer would interview vocational counselors and formet 
Students* employers. ^ 



' Pilpt Testing the Evaluation Model 

In an effort to identify any unforeseen problems inherent in the 
•design and proposed operational pxocedares'-of this evaluatiT)n m'odel, 
a pilot test was copducted at one of the state schools/state hospitals. 
The institution selected for the pilot test was conducting pilot pro- 
grams in vocational education for the* handicapped which closely 
replicated the offerings of the seven institutions identified for 
inclusion in t«is study. The evaluation model pilot test utilized thfe' 
exact data collection and analysis procedures that were to employed 
in evaluating the pilot programs in vocational education for the 
handicapped at the- other seveu institutions. 

Following the data collect^ion process, the A^isitation "t&anv 
members, along with an administrator from the pilot test institution, 
met to discuss the data collection procedures and review the instru- 
ments. Pertinent" suggestions, ^offered by this group, relating to 



procedural changes and inherent alterations were subsequently incor-" 
porated into the evaluation mod^l. 

' ' On-campus Visitations 

The visitatioh schedule, arranged by personnel in the Division of 
Occupational Pesearch and Development, Texas Education Agency,' called 
for on-campus visits to the seven state schools/state hospitals to- be 
made between March 17, 1975, and April 18, 1975. Prior to the insti- 
tutional visits, a letter was sent to the superintendent of each state ' 
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school/state hospital specif ying* the actual date of the visit and 
outlining the general procedures to be followed by the evaluation team. 
Because this letteij originated from the Texas Education Agency,, the* 
individual representing the Division of Occupational Research and 
Development was the team spokesman. 

ThB on-carapus visitationSj^which took from four* to six hours to 
complete at each institution, followed the format outlined below: 
— — A~meetujig-,\d^tlLJLhl. ^ of th e institution for 

purposes of: ^(a) exchanging introductions^ (b) discussing 
the purpose of the evaluation and the^ proposed disposition 
of the results, (c) explaining the procedures to be followed 
during the visit, and (d) ident^ifying the personnel to be 
interviewed, 

2. The actual collection of the data through personal inter- ^ 
view3 and prescribed observations • ^ 

3^-Vlsttatlon team meeting at which time the data collection- ' - 

instruments were reviewed for completeness and general 
observations discussed • - _ 

4, A second meeting with the superintendent for the purpose of 
* , discussing general observations made by the visitation team 

members • 

, Generally, th6 Director of Education and the Vocational^ Counselor 
were interviewed in their offices while the program instructors and 
students were interviewed in their actual classroom/laboratory setting. 
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A minimurorof three students per prograVwere interviewed. , Former 

students^ employers were intetviewed by telephone. At each institu- 

« \' 

tion^ the names of former students ^nd their employers* were obtained 

^ . i . . 

either from the institution counselor working directly with the pro- 
gram or from a Texas Rehabilitation Commission counselor located at" 
the state school/state hospital* A minimum of two former students' 
^employers were interviewed for each program and, when possible, the 
interviews were made at the time of the institutional visitation. 

Data Analysis and Processing 

Data analysis requirements of this evaluation model necessitated 
the us^e of two computer programs. One program was required to calcu- 
late the goal Weights, and a, second program was required to calculate 
an institution's objective achievement rate. Both programs were 
written in FORTRAN IV for a WATFIV compiler. 

— -^Thevcomputcr program-developed to calculate the goal weights 
(Appendix C) reads in as data input the importance s^nd effort ratings 
assigned to each goal statement by the director of education at each 
institution. Printed output fo^" this program includ^ed not only the 
calculated "goal weights but also the range, mean, and standard 
deviation for each rating scale. 

Data input for the second program (Appendix D) , which was 
utilized t</, calculate an institution's objective achievement rate, 
included the goal weights that were calculated In the preyious 



program and the ratings used to qualify ea6h interview question and 
prescribed observation. Printed output for this program included the 
nameof the institution and the title of the vocational education 
program; and, for each objective, the goal numbers, the goal weights, 
the number of questions relating to each goal, the sum of the ratings 

assigned to each goal, and finally, as a percentage, the institution's 

\ 

objective achievement ^rate. 

To, simplify the data coding process, a .special Data Tabulation 

. • . ; ■ ' 

Sheet (Appendix E) was prepared. This sheet was utilized to organize 
the data into a form suitable for keypunching. Coding procedures 

i ^ 

required that a Data Tabulation Sheet he^ completed for each program. 
Because the data contained in the Director^ o<f Education Questionnaire 
and the Vocational Counselor Questionnaire applied to all programs in 
a pdrt;icular state school/state hospital, these data.were recorded 
identically^ on each program sheet for a given institution. In situa- 
tions where more than one questionnaire was completed for a particular 

group (vocational students and former students' employers), the aver- 

\ * ' « 

age racing for each question Was entered on the Data Tabulation Sheet. 
Gen^^ally^ tt^e data generated by this evaluation model would not 

require analysis bfeyond the capabilities of the two above mentioned 

• ' \ 

programs. WiXh the determination of an institution's objective 

. • \ 

achievement rate^and the analysis of . each goal, sufficient evidence 
would; be present tb judge an institution's rate of success in meeting 
the vocational jiduca\ion needs of its clientele. However, for this 
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particular study, it was doomed m*oossnry to dotormino tho oxtnU to 

which the state schools/state hospitals, as a conj.\lom6rato r,roup, 

were achieving the specified goals and established obfjectivos, 

* 

To obtain an analysis of the group, a statistical routine 
described by Veldman (1967)- was utilized* This routine. Distribution 

Q 

Statistics* and Standard Scoring (DISTAT) , provides the following 
descriptive statistics: (a) number of responses to fehe variable, 
^(b) arithmetic mean, (c) standard deviation as a population parameter, 
(d) standard deviation as a Jjamplc -statistic, (e) standard error of 
the mean (population- parameter) , (f) standard error of' the mean 
(sample statistic), (g), sum of all scores, (h) sum of -all squared 
scores, (i) critical fatio for skewness, and (j) critical ratio for 
kurtpsis. Data input' fot this program' consisted of the institu- 
tions* objective achievement rate for each program and the insti*- 
tu^tions' average accomplishment of each goal statement, i,e*, number 
of cfuestions per goal divided, by t^e sum of the ratings given those 
goals. Both the institutions' objective achievement" rate and the . 
institutions' average goal accomplishment were obtained from one of 
the computer programs written specrtf ically f or ^Chis evaluation model. 

_ , - — 

Although it was possible to take the objective achievement rate * 

' ' ' ' ! 

directly from each printout, an additional print statement had to be 
added -to obtain the. average goal accomplishment. All data processing * 
for the above analyses was jierformeid" by the te5^as A&M University Data 
Processing Center ^Osing an IBM 360/65 computer system. i - * 



^ ' Sunimary "/ \ 

Based on theoretical 'post-iilates derived from the writings of 
evaluation theorists, an evaluation model capable of yielding 
quantitative measurements of ah educational institution's succesS in 
accomplishing (established objectives of vocational education was 
developed in this study^ This mcydel, which evolved from a skeletal . 

outline of basic evaluation strategies .developed by The^enter for 

■ y " " ■ ' 
Vocational and Technical Education at The Ohio State University, was 

• ' .■ ■<-■ 

field tested by conducting an evaluation of pilot programs in voca- 

tional ^education for the handicapped in selected state- schools and 

state hospitals in-Texas^ ' ^ , 

in this study, primary attention jwas directed towar/d the identi- 

fication of goals whidh were capable^f interpreting the -established 

objectives- of vocational education and the formulation of a data base 

with the capacity to yield reliable^ information relative?- to the extent 

to which the goals ,were achieved. By utilizing a systematic procedure- 

Co identify the goals and define the data base, it was determined that 

measurabl.e indices of program effectiveness could h,e acquired from six 

sourqes. These sources were* identified as institution administrators, 

' program instructors, vocational counselors, current students, toira^x 

studei^v^s* employers, and ^evaluation team members. It was also deter- 

mined that the most efficient method of obtaining, the desired informa- 

tion from these sources would be through the use of personal interv.iews 

and prescribed obseJtvationls. Thus, utilizing prepared data cpllection 



J 

instruments, an evaluation team proceeded to test llic ovnluaLion \\wk\k}\ 
/via on-carapus visits to selected state schools/state hospitals. 
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CHAPTp III 

...■•(■■, 

FINDINGS - 

The findings reported in this chapter are addressed to the two 
Contributing objectives 'of the study,' namely: 

1. To obtain a relative measure-of ^elected state school^ V 
state hospitals' succ'ess in Lpcomplishing established 



objectives o^ vocational education. 



evaluating pilot programs In vocational education for the \ 
^handicapped. ' * 5 
Consequently, this chapter has been subdivided into seven 
sections. The first five sections preset findings relative to the 
formulation of the goal weights and the success experienced at the 
various state schools/state hospitalf? in accomplishing established 
objectives of vocational education. The* sixth section'presents 
evaluation team members' observations relative to the testing .of Ch^^ 
evaluation model as was accomplished by utilizing the model and its 
related prpcesses to determine the degree of success experienced at 
,t he "various 4-i^stitiitions in accomplishing the established objectives 
of vocational ed ucation. A summary of ;all findings _^nd observations 
are then presented In a final section. 

i 

Findings Relative to the 
Goal Statement Ratings 

e 

Weight factors, ^derived from the importance and effort ratings 
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-assigned tho goal stntoments, were utiily.ed In the calculation of the 
objective cichlevement percentages for the purpose ot permitting tho^o 
individuals who were responsible, for the conduct of the pilot progrnms 
to provide input into the evaluation proceedings. These weight 
factors allowed various go^ls to contribute proportionate amounts to 
the objective achievement percentages, i.e., a goal with a weight 
factor of 8 contributed proportionately more to an objective achieve- 
ment percentage thaij' did a goal with a weight factbr of 5. Thus, the 

-vairi^us-welght~-facte£aix4pxfiattntr't^^ 

expressed in the g^al statement relative to the pilot programs in 
vocational education for the handicapped as conceived by the directors 
of education at the state schools/state hospitals Included in this 



study. 

%^ , ■'^^^ I^^^^OTS, as calculated from the importance and effort 



ratjings assigned the goal, statements by the directors of education, 
are displayed in Table 3. ^ Although the weight factors cpuld have, 
rangecj' from 1 to 9., their range only extended from 5 to 8 indicating 
that the directors of education viewed the concepts expressed iji the 
goal statements as significant to the pilot ^programs |irterms of 
desirable vocational education program developments. * In fact, nine 
goals received a weight factor of 8, eleven goaJs received a weight 
factor ofWl, three goals received a weight factor of 6, and only one 

goal received a weight factor of 5.. \ 

\l / ' ' \ 

In liite with a cited limitation of this study, i.e., those goal 

statements Which more nearly depicted the operational procedures of 



4.6 



V ' ' ' ^ Tabl^ 3 



Weight Factors as Derived from Importance 
and Effort Ratings of the Goal Statements 



'Goal 
No.. 



doal Statement 



Rating 



Mean . S.D. 



Weight 
Factor 



l«Cj| An institutiqn offering vbca- 
I tional education should possess 
a statement of philoiiophy whicli 
, reflects the institution's pur- 
pose and objectives relative to 
meeting the needs of the^stu- 
' dent-d-ientele^; — - 



Importance 
Effort 



4.57 0.53 
2.86 1.21 



1:0'^^ The/institution should possess 
a financial plan which is ade- 
quate to assure the quality and 
continuity of the vocational - 
education programs. 



Impor tance 5.00 0.00 
Effort '4.29 1.11 



:03 The institution should ^possess 
formalized future plans for 
vocational education* 



1:04 The institution should utilize 
a general advisory committee in 
planning for vocational educa- ^ 
tion and various occupational 
' advlsoty committees to assist 
in developing current curricu- 
lums for each program. 



Importance, 
Effort 


4.57 
3.14 


0.53 
1.21 


7 


is 


Importance 


3.71 


0.95 












5 




Effort ■ 


2.71 


1. 38 







2:01 General education personnel, 

professional personnel, and vo~\ 
cational education personnel 
should work together to organ- 
ize their offerings and- sen- 
vices in relation to the - , . 
vocational education program. 



Importance 4.71 0.4^ 
Effort 3.00 1*63 



2:02 Specific program instruction 

should be based on written per- 
formance objectives which were 
Ceriv<isd from an analysis of . 
required occupational compe- " 
tencles »- 



Importance 4.14 0.69 
Effort \ 3.14 0.90 
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Table 3 - Continued 



Goal 
No. 



Goal Stataaent 



Rating 



Mean S.D. 



Weight 
Factor 



2:03 There should be a continual 

analysis and subsequent Instruc- 
tor updating of the occupational 
competencies that .need to be , 
taughl for a specific occupa*- 
tlon. 



Importance 
Effort 



4.86 0.38 
3.86 1.35 



:04 Consumer education should be Importance 4.29 l.Il 

Included as an Integral part of > 
, the total vocational education Effort 3.57 1.13 

program. ^ 



2:05 Employers-employee relations Importance 4.86 0.38 
should be taught Integral 

part of the total vocational Effort 3.43 1.62 
education' program. 

,2:06 Students enrolled In an effec- Importance / 4.86 0.38 
tive vocational program should 

show evidence of change and Effort 3^71 1.38 

growth In attitudes. - 

3:01 Students with reco^cllable *edu-r Importance" 4.57 0.53 
catlonal deficiencies should be 

encouraged to enroll in the Effort 3.71 1.89 
vocational program. ' . 



3c 02 Specialized remedial Instruction 
should be made avall^able to .stu*- 
dents who have educational 
deficiencies which act as seri- 
ous barriers to successful pro- 
gr^ completion or job ]{)lacement. 



Importance 
Effort 



5,00 0.00 
4.14 1.57 



3:03 The potential number of> students Importance 4*71 0.49 
the vocational programs' c^n sue- * 

cessfully accommodate should be Effort 3.43 1.51 

enrolled. 

3:04 All vocational programs within Importance 4.71 0.49 
the institution should be of- 
fered -^on a twelve-month basis. Effort - 3.29 1.70 
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Table 3 - Continued 



Goal 
No. 


Goal Statement 


Rating 


Maan 




Weight 
Factor 


3:05 


The institution shpuld offer a 
structured career guidance pro- 
gram which enrolls all voca- 
tional students, ^ 


Importance 

Effort 

J) 


4.00 
3*^14 


' 1.00 
. 0.90 


6 



4>:01 The instructional practices^ 

carried'^on in the classroom and 
laboratory should provide stu- 
dents with simulated ybrk expe- 
riences reflective of^what 
would be expected of them in a 
jigag e- ear r^.ing„,s,it_u ajt.ipn.. 



Impor tanc e "4.86 0 . 38 
Effort 4.29 1.11 



4:02 Major equipment and machine - 
acquisitions should be commen- 
surate with the program objec- 
tives and when installed* the 
items should be complete With 
all salfety devices and placed 
oh a' preventive maintenance and 
r eplacement schedule • 



Impor t anc e 4.86 0 . ^8 
Effort 3.86 - 1.35 



4:03 The vocational programs should 
be housed in adequate facili- 
ties that are well managed and 
free of health > fire, and 
safety hazards. 



Importance 4.86 0.38^ 
Effort ' 3.71 1.38 



4:04 ' The institution should possess 
a formalized procedure for plac- 
H ing both graduates and non- 
graduates in an employment sit- 
uation that is In or directly 
related to the area of their 
preparation. 



Importance 4.86 0.38 
Effort . 4.00 1.00 



8^ 



4:05. Students in the vocational pro- * Importance 4.71 0.49 
grams should be involved ""'in 

cooperative or other out.-o£- * Effort' 3.86 1.07 

class work* e^^periences. > ' , . 

4:06 The .vocational instructional importance 4.71 0.76 

staff members should be certi- ' \ , ' 

fied in the areas in which they ' EiE^fort 3.29 1.50 

teach anc^ Se evaluated in terms 

of their .^aching proficiency. > I " 
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Table 3 - Continued 



Goal Weifiht 
Goal Statement Rating Mean S.D. p^^^j. 

4:07 Vocational instructors shoulci Importance 5.00 0.00 

use affective teaching methods, ^ \ ^ ^ 7 

procedures, and instructional Effort "3.29 1.38 

materials in the conduct of the 
teaching-learning process. 

4:08 The Institution shouf(l possess Impottance 4.57 0.53 

a formalized follow-up proced- ^ 7 

ure which yleld(^ infotlaation Effort 3.71 1.38 

relative to the adequacy, appro- 
priateness, aT\d effectiveness of 

jthe^JV-OCAtionaLdBducatlon prar^- 



grams. 



4:09 . An institution offering voca- loiportance 4.86 0.38 

tional education should conduct ^ 
. an annual. self-eval\iation for Effort 3.00 1.53 

" ' the purpose- of identifying 
strong and weak points as a 
basis fox*- up-grading the pro- 
grams. /> , . 



the pilot prograihs probfitbly received highej ratings (p; 10); Goal 1:04,, 
which dealt with the appointment and utilization of advisory commitL...es^ 
did receive the smallest * weight factor X5) and the lowest average , . 
achievemeht score (5^^percerit^ as depicted in Tables 3' and 4, respec- 
tively. Also, seven of, the nine goals which received weight factors of 
8 also received achievement percentage scores in the 85 to 95 percent 
bracket. . .. , - 

The directors of education at the various institutions included 
in this study rated, the importance of the 'concept '=»vpr^8sed in each- 
goal statement relatively high on the 5-point, Likert-type ^rating scale 
as only one out jDf, the 24 goals had a mean rating of leas than 4.00 and 



three goals had a mean of 5. 00* The standard deviations of less than 
1. 00 for 22 out -of the 24 importance ratings also Indicated that the 
directors of education substantially agreed that the cortcepts expressed 
in the goal statements should become .a functional part of the pilot 
programs in vocational education for the handicapped. As noteci by the 
larger standard deviations for fche effort ratings, less agreement 
existed among the directors of education as- to th^ degree of effort 
required ,to overcome obstacles such as cost of implementation, staff 



resistance, and lack of facilities and equipment that may have 
hindered the implementation of* the expressed concept into the pilot 

n 

programs. However, the effort rating means, which were substantially 
smaller than the importance rating means for each goal, indicate that 
any obstacles could be or have been overcome. In fact, only three out 
of the 24 goal ^statements jeceived effort rating means greater than. * 
4.00. ' . ^ ^ ' * 



Findings Relative to 
Vocational Education Objective 1 , 



Objective 1 addresses the concept of providing vocational educa-, 
tion to those indiv-iduals who can benefit -from the instruction by 
becoming better qualified^ to enter the world of work. In this evalua- 
tiort model, providing vocational education is interpreted to mean more 
than simpl^y conducting skill development classes. Providing vocational 
education also means creating an institutional atmosphere that is 
conducive to the continual growth,^ development, and further refii>ement 
of the vocational education programs. Thus, the 'goal statements 



associated with Objective 1 were Intended to exWplify various aspects ^ 
of a favorable institutional atmospherp; and, t\e degree to. which 
Objective 1 and its concomitant goals were fulfilled (Table 4) was 
interpreted as an indication of^ the various institutions' Success in 
creating this atmosphere, c 

Although the Objective 1 achievement seores ranged from 44 to 
90 percent (Table 4), the average achievement scorelwas 75 percent 
which indicates that the state schools/state hospitai^ls included In. 



tKTi^sTudy were sefl o usly altempriug Lu c r eate an Institutioha l: — " "~ 

atmosphere conducive to the continual growth, developtnent , tod refine- 
ment of the pilot programs in vocational education^ for^ the handicapped , 
A significant contri|)utor to this achievement score was the 92 percent ' 
average achievement score for Goal *1: 02, This substantial achieve- 

' ^ 

ment score indicates that the special funds provided for the pilot 
programs in vocational education were controlled through^^ accountable 
financial management sysftems which, met the approval of mast individuals 
involved, . \ 

\ 

As represented by the range of achievement scores relative to 
Goals 1:01,^1:03, and 1:04, the degree to which the seven institutions 
included in this study fulfilled these goals varied consideVably, A 
statement of ^philosophy which reflects an institution's purpose and 
objectives ^relative to meeting the vocational education needs of the 



student clientele was produced at each institution (Goal l;Oi). 

. • • • - • ■ • 1 ' 

However, in several instances ft was not clearly evident how the 
objectives were to be realized or that they were based on an analys"is 
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» Table 4 ■ . • 

Achievement Percentages Obtained by the State Schools/State Hospitals 
Relative to Objective 1 and Its Aaaoclated Goals 



Objective 1 

f 

Goal 

No* 



To provide vocational education to youth and adults who will 
be entering the labor force and to those who seek to upgrade 
their occupational competencies or learn new skills. 



Goal Statement 



Number of 
Out copies 



Sange of ^ Average 
Achievement Achievement 
(Percent) (Percent) 



1:01^ An institution offering voca*^ 
tional, education should possess 
a statttent of philpioph/ which 



14 



38 - 91 



IrefleetsTRe inst - 'iSiglocT i pur - 
pose and objectives relative to 
meeting the needs of the stu- 
dent clientele* 



71 



1:02 The institution should possess 13 


77 - 100 


92 • 


a financial plan which is ade-.. 






quate to assure t^ie quality and 






continuity of the vocational'^ 






education programs. 






1:03 The institution should possess 6 


18 - 100 


75 


fdrmallaed future plans for 






vocational education. 







l:Q4 The institution should utilise, 
a general advisory committee in 
plaunlog for vocational educa-* 
tlon and various occupational 
advisory^ coomit^ees to assist in 
developing current curriculums 
for, each program* 



11 



0 - 97 ^ 



Objectivi^l Achievement 



44 



44 - 9b 



54 



75 



of manpower needs or Job oppor^tunities available to the clientele; the 



institution was expected to serve. Eyidence qjt future plana for voce- 
tional education (Goal 1:03) at the state schools/state hospitals varied 
from the near absence of iay future"^antflRig to"fo^^ 



ERIC 



developeil through group meetings, which included lists of ' priorities 
and anticipated budgets. However, as indicated- by the average achieve- 
ment score of 75 percent for Go^l 1;03, most institutions did possess . 
definite future plans for vocational education. 

The wide range of achieveihent scprns and the average achievement 
score of 5A percent for Goal 1:0A ^indicates that vocational education 
advisory committees a^e not us^^exlTerislVeiy^ in the state schools/ 
state hospitals included in this study. Although it is stated in 

each local education ^gency "will establish local vocational advisory 
committees to provide^ advice relating to the asse^ssment of vocational 
needs and- planni^^, conducting, and evaluating the quality and effec- 
tiveness of vocational programs, services, and activities" (Texas ^ 
Education Agency, 1973, pi 72), this haw not been the case in a number 
of state schools/state l^spitals. ^However, eithef a general advisory^ 
committee ot -specific program advisbry committees had been appointed 
in a majority of the institutions. In many cases, though, evidence 
was lacking to suggest effective utilizatl^on of the advisbry committees 
as was noted that the committees seldom ^net, minutes of the meetings 
were not f'iled, and ,the institutions* responses to recommendations 
made by the advisory committees were not recorded. 

' Findings \elative to > * . ' 

, , • Vocational E<^c^t*idn Objective '2 ' ' 

General education, often defined as"^ that ^education which allows 



an individual to acquire the ability (and confidence in that ^ability) 



185 



to cope with one's environmerit in today's society, must be an^ntegral 

\ 

part of a well-planned, well-eKecuttd v()cational\ educat4,on program* 

\ 

It is assumed in thl,s evaluation model t:hat vocational education and 
'general education are complementary fco and necessary^ for each other* 
The totality of general e4ucation and vocational eduction received 
by each student will enable the individual to cope 'confidently and 
Siiccessfuliy with his environment.. Objective 2 and its tX^ricomitant . 
goal' statements (Table 5) are addressed to ^the expected rel^^tionship 
"h et:wegtr-^oca±±ana"l^^ ndrT geneTa'|-eduo^ — — — " 



As evidenced, by the average achLev*Jment score of 83 percet^t for 
Objective 2 (Table 5), it is apparent tliat an effort was being irtade 



at the state schools/statje hospitals included in this study to con^- 

pleraent education for earning a living with education for living.^^ ^ 

Thi^s point was further substantiated by the average achiev^ent scor^^ 

of 94 percent for Goal 2:01 which ladicated that the. general educatioa\ 

» • \ 

professional, and vocational education: personnel at each institution 

Wad cooperated in providing their respective services and offerings in 

relation to the vocational education ptogranTs* 'However, in a number 

of instances the program instructovH" expressed doubt as^ to whether a 

number of students placed in the various pilot programs could^ actually 

succeed in the .occupations for Which in.struccion was offered.* ; * 
* ■* ^ "J 

' An analysis of the outcome ratings which contributed to the 

average achievement-score of 88 percent: for 66al 2:02 revealed that 

specific program instruction in most. ca>5es was based on written 

performance objectives • It was not app^irent, though, that the students 
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Table 5 



Achievement Perccntai^es Obtained by the Slate School t^ystate Hospitals 
Relative to Objective 2 and its Associated Goals / 



Objective 2 To' provide comprehensive vocational education whicl^ 'relates 
general and yocatipnal educatipn offerings to the vocational 
objectives of students* 



Goal 
No. 



Goal Statemeiit 



2:01 ^General education pei 
* professional per 



^ Range of ^ Average 
.Number of Achievement Achievement' 
•Outcomes (peroent) (Percent) 



sonnely 
and vor 



10 



80 - 100 



94 




2r03 



sronndy , 
cationai education personnel ' 
should work tOgether*|to organ;|^ze 
their offerings and services in 
relation To the vocatjional edu- 
cation program* ' a 



Specific program instjruction ' 
shc>uld be based- on wrjitten per- 
formance objectives which were 
, derived from an analysis of re^ 
- quired occupational cpmpetencies » 

There should be a continual ana- 
Irysis and^ subsequent instructor 
* updating of .the occupational^ 
competencies that neec^ to be 
taught for a specific occupation* 




- 100 



88 



- 90 



77 



:04 Consumer education should be 

included as an integrall part of / 
the total vocational elducation /' 



0 - 100 



56 



program* 



— r— ^ r r — 

2:05 Employer-employee relations 

should be taught^ as an; integral 
part of the total vocational 
^education program* 


...^ 1_9 _ 


64 - 


99 

i 


^ 88 


t 


2:06 Students enrolled in an effec- 
1 tive vocational program should 
, ' show evidence of change ancl 
growth in attitudes* 


- 10 


] 87 - 


10,0 

> 


\x^93 ' , 




— Objective 2* -Achievement 


' 6'4 


- 72 ^ 


-97 


- -8-3- 





ERJC 
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understood the objectives or that dtita wore available to inaicate 
individual student progress toward the accomplishmenLt of the perform- 
ance objectives*. The appropriateness of; the performance object^^'es, 
as revealed by t|ie individual progi:am instructor's efforts to include 
instruction on new and emerging occupational competehcies ii:>rfo the 
plJot ,programs in vocational educat Lon,\7;us reflected in Goal 2:03* 
An analysis of the outcome ratings relative to tTjis? go^al fdisclosed 
the fact that , most vocational program ln%structors have ;had recenr • 

^x p e r ■ lences tn— fe^hei- r - area s H^^^p^^^a^^-gatA^n^^^jlthou^h-^ 



large majority *of the* instructors did not holdv memberships in profe$5- 
sionai organizations relative to theit toachlng fields, ^) 

The* wide range of achievement scor(2^^ relative to Goal 2:04 • 
fO - 100 percent) indicated that consunio.r education was an integral ^ 
part of some vocational education progr.hns and comple.telj neglected 
in others. GoaL 2:04 was included in tln,s jevuluat iqn model because 
many J'eaders. in the field of vocationaJ iHiucation are or the opinion 
that a vbcational student is not adequately prepared for employment 
until he/she has gained some understanding of the workings, values, 
.and institutions of. the American economy (fivans, 1971) • 

Goals 2:05 and 2:06, which addressed the two closely related - 
concepts of etppioyer-employee relations an^ development of student 
attitudes, were both substantially full lllcid as indicated by the 
respective average achievement scores of 88 and 93 percent. Although 
^many of the students who were interviewed were not cognizant of having 
received instruction relative to employer-employee* relations, 



vocational program insLriu-'lors tUsplayoil ovUloino of hnviii^ Inchuled 
such instruction In the pilot proj^nmis^ Also, ionnor sluclonts' 
employers who were interviewed were highly ^complimentary o£ the 
employer-employee^ relationsh^Lps they had "experienced with former 
students of the pilot programs in vocational education for the 
handicapped. • . ^ j . 



Findings Relative (o ^ 
Vocational Education Objective 3 



I ^Objective 3 concerns making vocational education available to 
!hs many people as possible. This concejv^ does npt mean that students 
i^hould be compelled to make final,, unchangeable career' d,ecisions ; 
instead,^ it means that schools should provide a setting which encour- 
'ages vocational decision making. It is assumed in thi^ evaluation 
model, as illustrated by the goal stateii)ents relative to Objective 3 
(Table 6), that students can fhake rational vocational decisions if 
they are informed o^ occupational opportunities and if they are given 
pr:)per support and encouragen;ci;;t. This, of course, requires a Career 



guidance prbgram that is based 



on student needs and a vocational 



education program that is capab^le of meeting these needs. 

In each. state school included in this study, the educational and / 
social needs of -the students were determined by an interdisciplinary 
team utiliiSing the ''Behavioral Characteristic? Progression" (BCP) . . 

.charts. Interdisciplinary teams were geneifally composed of a student s 

' ' . . . ' '* ' 

unit director and vocational program instructor as well as a psycholo- 

, ' : L- , / 

gist#, a social worker, a medical doctor^, and an individual representing 



Table 6 . i ^ " 

Achievement Percentages Obtained by the State Schools/State Hospitals 
Relative to ^Objective 3 and its Associated Goals 



Objective 3 To make vocational education increasingly accessibly to 
those who de3ire it. . 



Goal 
"No. 



Goal Statement 



Nuiaber of 

Outcomes 



Range of 



Average 



Ach i ev erne At Ac hiev eir en t 
(Percent) (Percent) 



3:01 StudMtB with reconcilable edu- 
cational deficiencies should be 
encouraged to enroll In the 

vocational program. 

« 

3:02 Speclarized remedla^l instruction 
should be made kyailable to stu- 
dents who have educational defi- 
cfencies which act as serious 
barriers to successful program 
completion. or job placement. 



3:03 The potential number of students 
the vocational programs can suc- 
cessfully accommodate should be 
enrolled. * 



3: OA .All vocational programs within 
the Institution should be of- 
• fered on a twelve-month basis. 

■ : t - 

3:05, The Institution should offer a 
structured career' guidance pro- 
gram which enrolls all voca- 
tional students. 



10 



h2 



34 



83 - 100 



88 - 100 



44 - 100 



80 - 100 



69 - 92 



96' 



98' 



\ 



85 



98 



85 



Objective 3^Achlevement 



66 



83 - 97 



93 



the school's recreation depart^nent.. By utilizing the, BCP charts, which, 
were composed of behavioral strands on the vertical axis and behavior 
characteristics within each strand on the horizontal axis, the inter- 
disciplinary team was able to plot an individual's- progress toward the 



acquisition of S|>i*cif ii- .ski lis and, when appropi-'intc , prescribe r^me- 



dial instruction 



The utilization of the- Intordiscip lina'^y toaji approach anc( the 

' / " 

BCP chafcts in the state schools and the use of the comprehensive 

' ■ ' - ■ ' ■ / 

treatment teams in the state hospitals contributed si^nif icaptly .to 

i • • ' ■ / 

the Objeci:ive 3 achievement score of 93 percent as depicted/ in 

Table 6» ^This objective achievement score was the result pt the* 

.exceptional average •achievement scores for each goal associated with 

• <- 



7 



Objective! 3 and is an indicator] of the success experienced at each 
institutipn in identifying and .appraising the social a^d educational 
needs of !the student clientele. The Objective 3 achiivement score 
was also indicative of the sutcess ^experienced at ea^h institution in 



providing specialised, remedial instruction for students who possessed 
^ identifiejd social and educational deficLencies that; may have acted as * 
barriers |to successful program compJeLLon or job placement/ Thus, the 
range of 
goals do 



achievement scores relative to ObjecLivep and its associated 
not represent the existence or non-existence of career guid- 



3tul 



/ 



ance programs based on student needs and vocational education programs 



capable of meeting those ne^eds, but represent the degree to which the 
'programs lappeared to be fulfilling the Identified needs of the student 
clientelel \ . ' " . 



The yar^iance in the degree to which student needs were being 

fulfilled lis portrayed by the relatively wide^ range of achievement 

1 \ 
scores for; Goals 3:03 and 3:05. \ These ranges in achievement scores 



were primarily due to (a) the pos^sibility that the range of vocational 



education programs was insufficient to ucet the expressed needs of the 
student clientele (Goal :):03) and (b) the limited * e oE vocational - 
information services (Goal 3:05). A sul)Stantial number of institu- 
tional personnel who were* interviewed expressed concern about the 
limited range of pilot programs offered ;it the state schools/state 
hospitals and many of the students who wt*re interviewed further 
substantiated this concern with" their expressions of doubt as to 
desiring future "employment in the occupations for which they were 
receiving Instruction. Directly relate*! to the students' expressions 
of". doubt was the limited use of vocational informatioa services. The 
analysis of the outcome ratings relative to Goal 3:05 r.^ealed that 
in a substantial number- of institut ion^; ^included in this study occupa- 
tional information relative to topics such as job characteristics, 
employtgent conditions, and worker ounl i I ications wa? \seldom collected 
and used as a means of assisrlin^', rstudenCs in 'naking 'Informed carreer 
decisions* 

Findings Relative to 
Vocational Education Objective 4 

In this evaluation model, to provide quality vocational education 
is interpreted to mean providing vocational e'ducation programs that 
arc both efficient and <ef f ective. Ip this case efficiency does not - 
refer to monetary cost per um t ,of 'instruction hot to the amount of 
student learning per unit of Lime; rather efficiency refers to the 
question, "Are students being taught the things they must know and do 
tQ become gainfully ^ployed in a recogni2^ed occupation, or is a 




large part of what they are learning only things they need to know in 

order to succeed' in the training progriim?" Several "efficiency Indi- 

«♦ 

cators" are inherent in the gonl statemc^nts pertaining to Objective A 

-* . ' 

(Table 7): .namely, the curriculum content aspect of Goal 4:01,\ the 
ease anH longevity of job placements as verified by former student 
follow-up studies, and the utilization of se.lf-evaluntion results. 
While program efficiency refers to curriculum content, program effec- 
tiveness, on the other hand, refers to the degree to which desired 
learner outcomes ^are achieved* Objective 4 goal statements relative 

to instructional practices and procedures, equipment and facilities 

I 

utilization, and instructional staff preparation are intended to 
.depict program effectiveness. 

The state schools'/state hospitals' sucv-zess in providing 'quality 
vocat-lona-l— eduea l^-on -is~dep4^-ed~4n--Xab44>-7~-.b^y^h e-jQbj^ctl v e ^-^uexage. 
achievement score of 79 percent. The range of achievement scores 
relati.ve to Objective A (66 - 96 percent) indicates that various 
institutions included in this study were more successful than others 
in providing quality vocational education programs for the handicapped 
Major findings which were primarily responsible for the Objective 4 

achievement score of 79 percent are reported as follows on a goal by 

ft ^ 

go^l basis: 

Goal 4:01 Evidence c!ol4eGt-ed-re4cV^iA^e— t^^t-hi^s^^ 
that student learning experiences in most of the pilot prograins were 
directly related to the skill and knowledge requirements of the 
occupations for which instruction W£is being offered. However, a 
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Table t . 

Achievement Percentages Obtained by the State Schoois/State Hospitals 
' Relative to Objective 4 and Its Associated Goals 



Objective 4 



Goal 
No. 



To provide quality vocational education which meets the vo- 
cational aspirations of people while being 'c6mpatible with 
employment opportunities* 



Goal Statement 



Number of 
Outcomes 



IRange of 
Achievement 
(Percent) 



Average 
Achievement 
■ (Percent) 



4:01 The instructional practices car- 13 
ried on in the classroom and*^ 
laboratory should provide stu- 
dents with simulated work expe- 
riences reflective of what 
would be expected of them in a 
wage-earning situation. 



49- 99 



85- 



4:02 Major equipment and machine " 10 
acquisitions should be commen- 
surate with the program objec- 
tives and when installed » the 
4.taBMi-<^ahould-bd» aompleta-with- — — — 



73 - 99 



89 



all safety devices and placed 
on a preventive mainteiiance and 
replacement schedule* 

4:03 The vocational programs should 
be housed in'^adequate facili- 
ties that are well mai?aged and 
free of healthy fire, ^nd 
safety hazards'* 

4:04 The instituj^ion should possess 

a formalized procedure for plac- 
• ing both graduates and non- 
' graduates in an employment sit-, 
uation that is in or directly 
x^elatad to t h e a r ea of-^hair 



16 



57 - 100 



88 



70- 100 



85, 



preparation* 



4:05 Students in tVie vocational pro- ^ 
grama should be involved in v 
cooperative or other out-of- 
• class work experiences* 



0 - 100 



67. 



ERLC 



TV 

* * 
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Tabio. 7 - Cont inued 






Goal Goal Statement • . " 
No. Outxomes 


Range of Average 
Ach iev em ent Ac h lev en t 
(Percent). (Percent) 




4:06 ^The vocational instructional 13 
staff members should be^certi- 
fied in the areas in which they 
te&ch and be evaliuited in .terms 
of their teaching proficiency. 


74 - 95 85 

r 




4:07 Vocational instructors should 14 
use effective teaching methods, 
procedures, and instructional ' 
materials in the conduct of the 
teaching-learning process. 


66 - 100 81 

« 




4:08- The institution should possess a^ ^ 10 
formalized followup procedure 
which yields information rela- 
tive to the adequacy, appropri- 
ateness, and effectiveness of 
the vocational education pro- 
grams. 


36 7 lOO* 71 




4:09 An institution offering voca- 8 
tiional education should conducxfc 


V 

9-93 55 




an annual self-evaluatii^n for 
the puri!K>«e of identifying 
strong and weak points as a 
basis for up-grading the pro- >" 
grams.- 






Objective 4 Achievement 96 


66 - SI/5' -79 ' ■ 




limited number of former students' employers who were interviewed i^adi- 




cated that the work produced by some of their employees wa#^only 


• * 


marginally satisfactory; thus, additional on-the-job supervision was 




required. At one institution included In. this study, no former ^ ^ 

* * » * \* 




stua^.tts could be located who were employed in 


an occupation for which 




they had received training; thus, it was impossible to obtain employer 

— '^y 


mc 


. '75 , 





verification that the occupational compat^-'ncies required of an indi- 
vidual to bcQome employable were actually being taught. 

Goal 4:02 Although several pilot programs were found to be 
""lacking somewhat in machines and equipmont, the major equipment items 
utilized at all institutions appeared ^to be in accordance with. the 
instructional objectives of the various programs and", in most 
*'in\3tances, representative of what is found in industry. However, at 
a substantial number of institutions it was hot determined that the 
final selection of the major equipment items was based, in part, on 

the recommendations of the various jidvisory committees; and, a number 
♦ 

of former students' employers, indicfited that their new employees 
required substantial retraining bef(»re i:hcy could become occupationally 
Competent. 

tJoal 4:03 Facilities housing the pilot programs at each institu- 
tion were well managed and relatively free of health, fire, and 
safety hazards. However, the 57 to 100 percent range in achievement 
scores for this goal provides an indication of the variance found in 
the adequacy 'of the facilities, AlLhouj»h many facilities had almost 
picturesque classrooiA/ laboratory areas » several facilities were found 
lacking work stations and materials and equipment storage areas, tt 
>'as also noted that a number of laboratory areas lacked an approved ^ 
system of color dynamics and appropriat(^ floor markings around power 
driven machines. 

Goal 4:04 Job placement functions at a majority of the institu- 
tions included In this study^ consisted of a cooperative effort between 

76 



the institution and the Texas Rehabilitatioti Commission. Generally, a 
Texas Rehabilitation Commission counselor, loratod at the institution, 
was primarily responsible .for placing students in employment situa- 
tions; thus, at most institutions job placement was not an integral 
part of the pilot programs and vocatiDn,\l education instructors were 
not actively involved in placement activities. The degree of success 
.experienced at the various institutions in placing s^budents in employ- 
ment situations that were in or directly related to the students' area 
of preparation appeared to be dep^endent upon the vocational education 
^progVams offered and the degree to which institutuional personnel who 
were directly. involved with the pilot programs were also involved with 
the placement acl^ivities. For instance, a significant proportion of 
the students who had been enrolled in i^ood service programs were 
employed in occupations d irectly related to their area of instruction; 
whereas a large number of students who had been enrolled in various 
other programs were not employed in occupations^ that were even remotel 
related co their area of preparation. Typical of a number of pilot 
programs was one where only three out of an approximate total" o^ 85 
individuals who had been enrolled were found employed in occupations 
which were in or directly related'-to, their area of preparation* Also, 
tliose institutions establishing job placement procedures in addition 
to those offered by the Texas Rehabilitation Commission appeared to 
have ^experienced greater success in job placement than did those 
institutions relying entrirely on the Texas Rehabilitation Commission 
Eor-^tudent placement in employment situations. 



Goal 4:05 Because .U.S. Departnu^nl X>{ Labor guldelines/f ail to 
interpret the difference between ski ll-<lovelopment training and wage- 
earning jwork, out-of-class work experiences at most institutions 
have been Severely curtailed. Although, as noted by the range of 
achievement scores for this goal'(0 - 100 percent), out-of-.class work 
experiences were still being ujLilizcd t •) supplement classroom instruc- 
tion.at a" few "institutions* - ^ 

Goal 4:06 An analysis of the outct^mc ratings relative to this 
goal revealed that all but, one of the various program instructors held 
State certification in the area in which th^y teach* Vocational pro- 
gram instructors at each institution were alsc evaluated in terms of 
their teaching proficiency. Thus, the range of achievement scores for 
-this goal (74 - 95 percent) represents the depree to which the person- 
nel evaluations included such practices as involving the program - 
instructors in the fo.nnula^ion of t-ic. poi sonncl eval»jation plan, 
planning in-service education on tho bar>..s of the peisonricl* evalua- ^ 
tions, and assessing the tapabilitii^s ob the >?ocatiorial staff members 
in terms of curricular needs. 

Goal 4:07 Because Goal 4:07 r ipifosonted numerous facets relative 
to teaching methods and procedures and instructional materials, find- 
ings pertaining to this goal ar^^^^sentud Xn the form of a listing. 
The range of achievement scores relative t;o this goal (66 - 100 per- 
cent) indicates that a program by .program variance existed in the 
outcome ratings; therefore, the following list represents general 
findings relative to Goal 4:07. These findings are': 



67 



a. luKlrmM lonnl imilorlirls woro iUH-uiMito in runUMit aiul 
rulLortrd current occupnL iona 1 knowlodgo and practices. 

b. Classroom and laboratory activities were , coordinated in ' 
terms of meeting the instructional objectives of the 

rxvocational education program^. 

c. 'A sufficient quantity and variety of tnuin'rcuc and materials 
were available to facilitate a multi-media approach in the 
instructional processes; 

d. .Student; learning activities were organized in such a manner 
that individualized instrucLion, when appropriate,^ was 

available to ail students. 

I? 

e. Supplementary instructional iiXK\s and teaching devices were,, 
utilized to provide Eor speci^jl interests or learning 
problems. 



f . Sufficient space and equipment were provided £or Students ' 
to pursue independent study. - * 

V 

% 

g. Written- "lesson 'plans were prepai?ed and real or visual 
materials were utlif.5:e(l with each unit of instruction. 

h. A satisfactory system of checking, servicing, and storing 
tddls and materials existed. 

i. Appropriate clothing was worn by the instruc4:ors and students 
whe n workin g in the laboratory areas. 



j. The reading level of written Instructional materials did not 
appear to be keyed' tc the reading competence of the students, 
k. Evidence of individual student progress toward acquiring job 



skills was not recorded. 

.1 



—1. • Activities sucJh as field trips and on-site visits were nojt 

used as a. method of instruction* 

in. Resource people from the communities were not utilized in the 
instruction-al processes. 

Goal 4; 08 As evidenced by the ran}',e of achievement scores rela- 
tive to this goal 1(36 - 100 percent), autempts to follow the progress 
of former students varied considerably from institution to' institution 
Former student follow-up records at several institutions were excep- 
tionally coniplete and did reflect information relative to 'the*adequacy 
appropriateness, aiijd effectiveness of the pilot programs. However, at 
a number of other state schools/stare hospitalp it was noted fhat 
former student foliow-up was not an integral part of the pilot pro- 
grams • Vocational education Instructors were not directly Involved 
with follpw-up^_f.ctiyitle8, and former students' employers had not been 
surveyed in an effort to assemble information concerning the effec- 
tivenesB of the pilot programs in preparing handicapped individuals 
for gainful employment*- ^ 

Goal 4:09 Evidence collected -^Indica ted that self-evaluations 

of the pilot programs were conducted at the institutions included in 

. — ' " -i 

this, study. However"", as reflected by th(^ average achievement score . 

of 5^ percent for this goal, a majority of the self-evaluations were 

V • ? " " 

conducted on a yery^ informal basis. Consequently, _it was generally 

found that final reports were not fiJed and vocational program 
Instructors and^ advisory committee membiirs were not directly involved 
in the self-evaluation activities* 
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Evn J nation Te^im fIeipt>ors' Observat I^nirr* Relat 1 vt» 
^•o tljio UtLilty of tho Hvaiualion Model 



As, a fiorm of met^-eyvaluaTFon, tho visitation team members, exrlu- 

1 * 

sive o^ this researcher* were asked to prepare a statement regarding 
their perceptions as to the appropriateness of .the evaluation model 

developed in this study in terms of assessing the (Effectiveness of the 

i 

pilot programs in preparing^ handicapped individuals! for gainful, 

i 

empl^^jn^ — These statements follow: . «>i 

It was my pleasure to have recently participated as a tnember 
of the team evaluating Vocational Education for the Handicapped 
Programs in State Schools ..a^id Hospitals ; which operate under the 
jurisdiction of the State D|epartment of Mental Health and Mental 
Retardation, i 

^ased on 17 years of experience in the field of vocational 
education, ±V is my opinion that tht-" ffihai results of the eval- 
uation, which I have read, present a true and accurate, picture 
of the "State of the Art" as it exists in the institutions I 

visited, > \ ' ^- > - 

Furthermore, it Is my belief that the model utilized could . 
Iiead £o a new and unique solution to the nationwide problem of 
eyaluating vocational education. One,- of the^model's strongest 
implicatxQAs in this respect is that jit couldi be equally ef f ec- 
t:^ve,'with very little modif icationt in all facets of vocational 
education including Agriculture, Hom<^ Economics, Industrial,, 
Distributive, Office, and 'Health prbj^rams for handicapped.,, / 
educationally* disadvantaged,^ and regular students at the m^iddle 
school, high^ school, and post secondary level of instruction^ 

It is my hope that this research can be further developed, 
validated, and utilized for the tulucationab betterment of all 
our/ Ration's youth. . ^ '/ ^ 

^ i . . 

T..I\. Jones, Chief Consultant 
' Department of Occupational 

Education and Technology 
' Texas Education Agency i 

• - Aust in, TX ' I 



r 



Although I' was unable to partitipaUe In the evaluation of al^l 
state schools participating in this project, I enjoyed member- 
ship on the visitation team and fnound the experience both pro- 
fessionally and educationally rewarding, ^ ^ 



The selection, of a design which utilizes' people who are know- 
ledgeable about vocationiil education and/ experienced in intetvi 
techniques to collect the data for t)ie/study is a strength^ 
Other strong points include: / . 



Administration of the data collection 'instruments is a rela- 
tively* uncoinplica^ed process, ' 4^ ' 1 ' * 

"■1 • . " . ' ' ' ' • 

The categories usually considered Important in vocational 
education programs (philosophy of vocational education, 
written Objectives, advisory councils, etc,) were addressed, 

) ' ' ' . ' ^ 

Except for the wording of gome questions in- the data collection 
instruments, no major weaknesses were obvious. The final reports - 
written for the two institutions T visited appear to be congruent 
with my Impjl^si^^lpris of the . Evaluation sessions, ^ ' \ 

^ X ^1 ' Paul Myers, Research Management 

^ ' Assistant ^ ; 

I • Divisioq-of Occupational Research 

and ' DeyeloppeinJ 
^ \ ^ Texas Education Agency'^ * 

Austin, TX I 



Although th^ model for evaluating the institutional A^ocational 
education programs for the handicapped appeared to be Related 
more to secondary and post secondary public school vocatiDnal 
education progranffe than .the programs Cor special populations in 
a residential setting, the findiji|;s, In m^ opinion, accjuratcly 
reflected the quality of these, progragis according to the stated 
goals and objectives, * ■ I 

I administered various questionnaires at two state sdhpols and 
one statfe/ho^spital. The instrumejits-were well d^signc^ |dnd easy^^ 
to administer. The summary repoxts revealed a wealth o^/ 'informa- 
tion that was relevant for pragtcjm, p Lannlng, developmerjtl , and. 
improvement, ^ , / ♦ 

A somewhat different type of -tjuj^&tlohnaire should be^idesigned 
for interviewing the mentally retarded vocational education 
student. Most of the students 1 interviev^ed were nof^ble to 
comprehend or accurately verbalize their -feelings concerning 
the value or' benefcit of 'the training they were recfjeiving. ^ 



\ 

In cofncluslon, ^1 f eel that the model is an appropriate instru- 
'.Hjeht fpr evaluating vocational education programs. 

, • / • ' • Louis Corenblith, Consultant 

' ^Rehabilitation Services 

' * Texas Department of Mental Health 

* , and Mental Retardation 

Austin,. TX 

/ • * 

, Acc^rding^to my per.ceptions of . the* thtee programs that I par- 
Jticiptted in evaluating, the evaluation model produced meaningful 
scores and especially constructive criteria^ The evaluation 

/roo<fel was valuable^ in' specifying' iDot'.h negative and positive 
aspects* of the vocational education programs visited. In addi^^ 

<tion, negative aspects of the programs /were evaluated in a 
constructive manner which should lead to improvement in deficits. 

^Howevery. another study should 'be' conducted In an attempt to 
evaluate* the goals and objectives* in terms, of their meaningfu].- 
neas to state schools Vstate hospitals* personnel and student^. 
This may 'lead to some significant changes in wording, answers, 
,ahd content of the qifestions. • Weights would possibly change also, 

' ' • ' * Stanley Fqdell, Ed.D», Consultant 

» Prograifis for the Retarded 
' , Texas DepartnieTllrljf^Mental Health 
, and- Mental' Ret3Tdati*on 



Austin,- TX 

1 . f 

The vocational program evaluation model utilized in evaluating 
.the vocational education programs for |th^ handicapped* in the 
state schools and hospitals in Texas lis. a well developed document. 
It is a corjprehfensive lev.aluation model that, in my opinion, would 
ljettei< serve as a model to follow iri Evaluating vocational educa- 
tion progtaras in. the public schools. The/type of clientele 'served 
by the state schools and hospitals differ greatly^, f rom students in 
public school programs. ^The nature o.f adminiattation of public 
schools compared to thatlpf- institutions also varies 'significantly. 
These factors and other jr^iables make it difficult to evaluate, 
the success or, failure of aiw program in these Si^.lect settings 
and -fior *these special populations. 

(Mrs.) Eleanor K. Mikulin,' Chief 

Consultant 
Special Education Programs 

Administration 
Division of Special Education and 

Special Schools 
Texas Education Agency ^ . 
Austin, TX ' * 



Your "rough drafts" follow the instrument format and give the 
desired information we were seeking. The interpretations of the ^ 
various items, base;d on the raw data, gave a good evaluation of 
each of the facilities. I endorse these four documents* / 

/ 

I Paul E. Williams, Consultant 

I ' Special Education Program 

Adrainistratdonr j 
Division of Special Educairion and 
■ • • Special Schools 

I ^ Texas Education Agency ; 

j Austin, TX 

\ 

Summary 

ft 

Trapoi^tance and effort ratings assigned the goal statements by the 

directors of education at the seven state schools/state hospitals 

included in \h±3 study resulted in goal weight factors that^ ranged 

from 5 to 8» Out of k possible range of 1 to 9 only one goal received 

a weight as small as 5, and 20 of the 24 goals received weight factors 

of either 7 or 8,^ Because these weight factors were calculated from 

the importance and effort ratings assigned each goal statement, they 

represented the significance of the programmatic concept expressed in 

the goal statement relative to the pilot programs as conceived by the 

directors of education. Therefore, relatively large weight factors 
* o 

indicated that the directors of education viewed the ^expressed con-; 
cepts as significant features of jhe pilot programs in vocational^ 
education for the handicapped. - ' * - 

The degree of success in accomplishing established objectives 
of vocational education experienced by the state schools/state 
hospitals as a group was reported in tlie form of average objective 
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achievement sc'ores. These scores , whic.h were a summary of the average 
achievement scores^ for the goals associated With each objective, con- 

■ /, ' ' ' '\ 

sistecl of 75 percent for Objective 1, &l percent for Objective 2, 

/ ! • ;■ \- 

93^ percent for Objective 3, and 79 percent for*' Objective 4. 

Objective 1 addressed the concept of providing vocational educa- 

' \ ■ ■ ' ! \ 

tfion to those individuals who could benefit from .the instruction by 
' \ ' I ^ - 

/ ' \ \ ^ 

becoming .better qualified to enter the woVld of wo^rk and was inter- 

preted by four gpal statements. Thes^e four goal statements, which 

dealt with inst;itutional statements of philosophy relative to voca- 

;tional education, accountable financial management systems, future 

plans for vocational education, .and the 'appointment and utilization 

of advisory committees received average /achievement scores of 71„ 92,. 

75, arid 54 "percent respectively. The average achievement, scores of 

71 percent for Goal 1:01, 92 percent for Goal 1:02, and 75 percent for 

N • > ' ' . ^ 

Goal 1:04 indicated thatr the institutions included in this study were 

seriously attempting to provide for vocational education by creating: 

an institutional atmosphere conducive to the continual growth, devel- 

opment, and further refinement of the pLlot programs in vdcatlonal 

education for the handicapped. Hoyever, the range of achievement 

SQores for Goal 1:04 (0 - 97 percent), in conjunction with the Goal 

! 

f - K . 

1:04 average achievement score of 54 percent, indicated that advisory 

1 

committees generally were not involved in the planning and Imjilementa- 
tion of the pilot programs.' 

' The substantial ayerage '"achievement scores for a majority of the 
goals associated with Objective 2 indicated that the students enrolled 



in the pilot programs at the various institutions were being provided 
comprehensive vocational educcJtion which related education for earnim 
a Jiving to education for living. Only Goal 2: OA, which had an aver- 
age achievement score of 56»percent, was 'not' part'icularly well ful~ 
filled; and, as revealed by the range of achievement scores for this 
goal (0 - 100 percent) consumer education, an aspect c\f education for 
living, was not included as an integral part of the total vocational 
education program at some institutions* 

Average</achievement scores of 88 percent for Goal 2:05 and 93 
- percent for Goal 2:06 revealed that instruction relative to employer- 
employee relations was being provided and that^a sincere effort was 
being made to assis£ the students in developing positive work attir 
tudes. The avera*ge achievement score of 94 percent for Goal 2:01 als 
indicated that the general education, professional, and vocational 
education personnel at most institutions had cooperated to organize' 
their offerings and services in relation to the vocational education 
programs. However, in several instance?; the program instructors did 

a 

express some doubt as to whether a number of students who were-piaced 
in the pilot programs could actually succeed in the occupations for 
which instruction was being offered. 

Making vocational educ'atioa increaj!?^ngly accessible to those . 
individuals desiring such instruction was the primary concept 
expressed in Objective 3. In this evaluation model that concept was 
interpreted as the identification of the students^ social €^nd educa- 
tional needs and the provision of the support*, encouragement, and 



instjruction required to fulfill thos^ncieds^ Through the use of inter- 
disciplinary evaluation teams and with the aid of the "Behavioral 
Characteristics Progression" charts, the task of identifying student 
needs was being accomplished in the state schools and state hospitals 
Included in this study**; and, as was evidenced by the significant aver- 
age archievement scores for the goals^ pertaining to Objective 3, defin- 
ite efforts were being made to fulfill those .identified student needs^ 
In fact-, three of the five goals pertaining to Objective 3 bad average 
completion scores of 96 to 98^^perqent and the other two goals had 
jscores of 85 percent^ - However, the 9PSKibillt:y that some institutions 
-were not offering a sufficient number and variety of vocational pro- 

ft 

grams to satisfy^the varied aspirations of the student clientele and 
the limited use^of vocational information services in several institu- 
tion^ widened the range of average achievement scores for Goals 3:03 
and 3:05/ 

- OViective 4 addressed the concept oi providing quality vocational 

* i 

education pro.grams capable of preparing individuals to enter the world 
of work in recognized occupations. Consequently, the goal statements^ 
associated with Objective 4 essentially -pertained to the operational 
procedures and instructional method^ employed in the conduct of the 
pilot programs in vocational education for the handicapped. Although 
the average achievement, scores for Objective 4 were relatively; con- 
sistent from goal to goal, the range of scores within each goal was - 
quite extensive. Thus, the degree of success experienced in fulfill- 
ing the goals varied from institution to institution. 



At a large majotity o.f the institutions included in this ^ 



tudy"7^ 



the curriculum materials utilized in the JLnatruct^onaJLpxoce^&es^asu/:: 

well as the skill development experiences received by. the students 
reflected the skill and knowledge requirements of the occupations for 
which instruction was being offered. Student learning activities 
were also organized in such a manner thnk individualized instruction, 
when appropriate, was available to all students. However, evidence 
of individual student progress toward acquiring job skills was gen- 
erally not recorded and activities such as field tri^s and on-site 
visits were not utilized as a method of instruction. It was also 
noted that the self-evaluations of the pilot programs were generally 
conducted on an informal basis. 

Facilities housing the pilot programs were usually quite adequate 
and the machines and equipment therein were commensurate, with the 
instructional objectives of the various pilot programs and, in most 
instances , ^representative of what would found in business or 
iildustrial situations. However, several former students' employers 
indicated tjiat their hew enjployees required substantial retraining 
before they could become occupatiorially competent. 

As was evidenced by the average achievement scores for Goals / 
4:05 and 4:08, job ^placement and former student follow-up procedures 
were not fully developed in several state schools/state hospitals- 
The Texas Rehabilitation Commission wa« to hav^ primary responsibility 
^or job placement and former student foi low-up (Phase III of the state 
plan for vocational education for the handicapped). However, at those 



institutions where job placement procedures had been established in* 
addition "to those offered by the Texas Rehabilitation Conimission 
greater success app^eared to be experienced in locating employment 
situations thdt were in or directly related to the students' area of 

V 

preparation than was' experienced at those Institutions where the Texas 

Rehabilitation Commission had sole responsibility for job placement. 

This same phenomenon seemed to apply to 1 ormer student follow-up, 

i«e., at those institutions where former student follow-up procedures 

had been- established in addition to the services offered by the Texas 

'Rehabilitation Commission more complete t<ormer student follow-up 
c * . . 

information seemed to exist than existed at those institutions where 

former student follow-up procedures werti not definitely assigned* 

Statements written by six individua.!s wh(» had asJsi^ted in the 
data collection and had read the final rt^ports issued the various 
institutions included in this study were utilized as evaluation find- 
ings relative to the 'design and operation of the vocr^tiona] program 
evaluation model developed in this study This group of individual^ 
found the evaluation model and its related processes capable of pro- 
tiijcing a comprehensive, accurate account: of what was" actually 
transpiring in the pilot programs* Tho data (collection instruments 
were found to be relatively easy to administer and the fact that 
individuals' rather than self-administerod d^ata coWection Instruments 
were utilized to collect fhe evaluative data was listed as an advan- 
tage. However, it was noted by the abo^'e mentioned group t)iat the* 
evaluation models and its related processes appeared to be more 
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applicable to the evaluation of secondary ^nd post-secondary public 
• , *" 

school vocational education programs, than to , programs for special. 

\ 

populations^ in residential settings. iV was also noted by one 
individual and insinuated^ by others that trie interview questions 
contained in the* "Vocational Student Questiont^aire*' were not appropri- 
ate for mentally handicapped individuals. 
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CHAPTER IV ' . 
SUMMARY, CONCLUSIONS, AND REC(»1MENDATI0NS > 
/\ / ■ Summary 

, 4 

The putpose of this study was^to develop, an evaluation model , 

C * I 

which could be utilized to determine the effectiveness of pilot pro- 

grams in vocational education for the handicapped in terms of meeting 

* 

established objectives of vocational education. This purpose was to 
be accomplished by fulfilling two contributing objectives, namely: 
1. To test the model and the related processes necessary "for 
evaluating pilot programs in vocational educatioti for the 
handicapped. - ^ 

2* To obtain a relative measure of selected state schools'/ 

state hospitals' success in accomplishing established ' 
^ objectives of vocational education. 

Following an extensive review of the lite/ature relative to 
educational evaluation, six postulates were formed concerning the 
expectations of a vocational program evaluation model, ^hese postu- 
lates, as listed below, constituted the framework for the development 
of this particular evaluation model: * 

1. An evaluation model must be capable of providing factual ^ 
decision-making information to a specific audience. 

2. An evaluation model must provide ^a means for fully describing 
all aspects of an educational program. 
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6.- Performance indices empioyed to determine program 



3. An" evaluation piodel must be capable of producing reliable 
> 

decTsion-making information without relying on an experi- 
mental design, * ' 

4. Evaluation models .utilized in education should be designed, 
• anid devieloped for educational situations. 

5. Data analysis procedures employed in a program evaluation 
model should allow for direct program comparisons. ' , ' 

vef^ 

ness should be composed of both process and product »data. 
Beqiifte The Center for Vocational and Technical Education at The 
Ohio State University (hereafter referred to 'as The "Center)^had ^ ~~ 
developed a system for state evaluation "of vocational education pro- 
grams which also appeared to be based on a number of the previously 
♦ 

Stated postulates, The Center systetn was utilized as a point of 
departure for the evaluation model developed in this study. However, 
the evaluation procedures developed by The Center^were designed to be 
implemented in an educational setting that differed substantially from - 
the state schools and state hospifals in Texas. Thus, apart from a^ 
similar skeletal outline of basic evaluation strategies, the processes 
and procedures employed by this model were germane to the pilot programs 
in vocational education for the handicapped. 

Similar to the evaluation model created by The Center, the model 
developed. in this study, has objectives, goals, and outcomes -as its main 
components. The vocational education objectives utilized in this study 
were formulated by The Center and represent the work of evaluation 



specialists, psychologists, vocational program specialists, and state, 
directors* of vocational education with* final validation by a national 
survey of state directors and head state supervisors of vocational 
education* Because these objectives represemt the concepts of leaders 
in the field of vocational education and because they are also suffi- 
ciently global to apply tO; any program at any level, they wene utilized 
in the evaluation 9^el .developed in this study.< The objectives are 
s^tated as follows (Starr & Dief fenderfer , 1972): 

1, ' Tn provide vocational education to youth and adults w^o will 
be entering the force and to those who seek to upgMde 

their occupational^mpetencies or learji new skills. ^ 

To provide poraprehensive vocational education which relates \ 
general and vocational education offerings to the vocationar 
' ' objectives of students* 

3. To*make vocational education increasingly accessible to 
those who desire it* • 

4* To provide ; quality vocational education which meets the 

vocational aspirations of people while- being compatible with 
employment opportunities, (p* 14) - 

However, the universality of. the vocational education objectives 

made them insufficiently precise t6 be, measured directly* Therefore, 

it was necessary tcr^ interpret each objective in the form of numerous 

goal statements which were sufficiently precise to be measurable. In 

addition to interpreting the objectives, the goal statements vere also 

designed to: , 

1, Produce a ;care of information which would yield an assessment 

,of the extent to which the established objectives of voca- ^ 

* 

tional education were being ^achieved* 
2* Reflect the vocational education Intents of thei state school^ 



, and state hospitals as^ stated^in their initial proposals to 
the iEunding agency. 

3. Reflect the program decision requirements of the funding 

«■ * 

agency. 

4. Provide a basis upon which oto make a full description of 'the 
intejits and activities of each pilot program In vocational 
education fox the'' handicapped.. v 

Jusf as there were numerous goals for each objective in this 

evaluatioti model, there also were numerous outcomes for each goal. 

The outcomes, which constituted the data base, were' derived through a ' 

systematic procedure* which consisted of the following four steps: 

1. Li'&ting each vocational education objective*.^ 

Listing under each -objective the goal stifttements that would 

interpret the specific objective. 

3. Listing under each goal statement the data elements (outcomes) 

4 required in order to determine if the gdals were being ^ 

«>. * * 

achieved. 

• 4.^ Listing under each data element the spUrce of the required 
information. 

T^hus, a direct relationship was established between the objectives, 
the goals, £Cnd. the outcomes and a form of additivity; was^develop^d in 
that (a) the extent to which ea-ch -goal statement ^was fulfilled was 
determined by the numbet of affirmative responses (outcomes) relative 
•to each goal; and (b) the extent to which each vocational education 
objective was accomplished was determined by the degree to which the 
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goals relative to the specific objective Were fulfilled, 'Consequently, 

by'first summing the number of? affirmative outcomes related to each 

'I " * « . - ' ^ 
y/go^l statement, and next, by summing the degree to which e^ch goa^ ^ ^. 



\ 

statement was fulfilled 



it J 



ecame possible to obtain 



I a re 



latlve roea-' 

/ 



sure of the success experienced at an injstitution in- accomplishing a 

/ 



/ 



particular vocational education objective. *' i 

I Two additional* factors incorporated into this additiv'^ evaluation 
model Were the weiglit factors assigned the goals and the qualified 
ratings given the ou^coJnes/* '^he goal weights were .derived from^ ratings 



/ 



assigned each goal statement by those individuals who yere responsible 
for the conduct of the progriams included in the evalohtion*^ On- a 

\i * ' r • ; ' 

5-point Likert-type scale, theise individuals rated ±he programmatic 



concept expressed in thejgoaj statement in terms of i.ts relative 

• . • / .\ ' . '■ . • 

importance to their respective programs and the degree of -effort they 

\ " . ' ' . * * 

would have to expend tb incorporate the expressed concept into the / 

; ^ ^ ____ 

programs under their jurisdiction, , Because all data -elements derived 

. • ' ■ , \ - " . . . . ; ' 

through the systematic ^procedure previously described (Step 3) 

■ • ^ ^' • , \- ' V. ' ■ ■■ 

culminated in questions and prescribed observations and because these 

\ ■ '\ * • 

questions and observations wer^^if f icult to answer ^±t\i an ^unequivocal/ 

"Yes" or "No," a provision was made to qualify each answer • This pro-^ 

\ . / 
vision consisted of a" rating scale \which was based on the degree of 

- / \ '\ ■ 

evidence utilized by the respondent \to support the "Yes" .or*"N©l!J 



ansver to the specific questlqn or prescribed observation. A dynamic 
mix of process and product type evaluative criteria was ajso developed 
in Step 3 pf the previously described procedure in that most process 



type data el^etjts* were given a products type' counterpar^t, i.e., If a 
process type question was developed for one group, verification of the 

! \ 

possible answer was sought by developing a prescribed observation or a 
product type question /or another group. i 

^At the completion of Step 4 in the previously described data base- 
developmeht procedure, six sources of information had bjeen identlfiecj: 
namely, directors of education, vocational program \insfcrtictors , voca- 
tional counselors, current students , former students,' employers, and 
evaluation team members • Al^so, at this point in' the \development .of the 

. e^^^f^tion model' and tts related process, it was deter^inec} that the 

/' n 

/ ' * ^ \ 

most efficient method of acquiring, the required information from the , 

' f ■ ■ , '\/ ■ ' 

^identified institutional personnel, students, and forme^r stpdents' 
employers Would be. through the use of personal Intervieis, Likewise, 
it was determined that the visitation team members could contribute to 

" - .1 ^ \ ■ ■ 

the data b^se mofft ef ficifently by making prescribed observations • 

/ '',P ^ ' ' I 

* F6*lliqjiiing,the development* of the data base and ,t*ie identification 

, . ^' ' , 1 

gf the' infor|nnation sources, appropriate data, collection instruments 
were prepare These instruments consisted of pne questionnaire which 
to obtaifi the'goa] statement importance and; effort ratings 

" .i 

Lou^ directors of education« an4 five different interview 



\ 

was uti*liz«d 



JErom the. var 



ques^^npalTiis which were utilized to, obtain the desiredi' Evaluative , 

\ - ' I y 

information' ;:rom the- previously identified i-nforraation slources—exclu- 
sive, of the ^rlsitation t^am members, \ Prescribed observations, which / 
were made bjj the visitation team members*, were included ^ the appro-/ 

■ ' ^ " • " I / 

prfate iJersonal Interview questionnaire. The contlenti validity of the 
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• data colle<*f6n Instruments was subsequently established |5yi a jury of 
experts,. In conjunction with the development of the data col/e(?tion " 

instruments, data analysis procedures wore also developed* The two 

» ' " *' 
. ^ . r • » . ' 

primary components of the data^^alysis procedures consisted of com- 

' ' ' ^ 

puter programs. One program was written and utilized to calculate 

\ 

"the weight factpr^: from-^.the goal statemcint importance and effort 

I \ ^ 

ratings while the second program was written aind utilized to calculate 
the institutional objective achieyemant scores on a program by progrkm 
basis. A pictorial summary ofjthe evaluation' model and 'Jts related' 
processes is depicted in Figure 2. " ' 

ibiiequent^to^the development of the evaluation model ind it^ 
relatetf^processes, test sites were chosen and visitation team members ■ 
were selected. Texas Education Agency Division of Occupational 
Research and Development, personnel were responsible foy'choosing the 
test sites and selecting the visitation team members. Seven state / 

/ / . ■ / r . > ' 

^ schools/state hospitals were selected for participation in the field 
testing of the evaluation model and its related processes while an 
eighth institution was utilized for pilot tepting tKe design and 
proposed operational .procedures of ^ the evaluation model. The visita- 

I 1 ■ ^ 

tion team selected consisted of four 'individuals who were representa- 
t'ive of vocational-education, the Texas Department of Mental Health 
and Mental Retardation, the Divlsioji of Ocdupatlonal Research and 
Devclopmc Texas^ Educ^ition Agency,- and the Di'vision of Special 

• / ' l' ' ^ 

, Education and Special 'Schools ; Texas Education, Agency. 



- ■/ /• . / 



'/ 

/ 



.,97/ . 



Ordtr of 0«vt lo pwn t 



Ord«r of Operation 



Objfecclvfc 



Goal, 



-^^ j Out come 2 | 

I Ouicomet | 



f 







Goal 

2 









^ OuCcome2 | 



Goal 



^ I OutcQme^ I 

[^Outcome2 | 

^ 1 Outcoacn | 



Rtlattd Proctss^ 




- four objectives are utilised which are universal in nature and express desired oucccno 
for vocational education in tcnaa of national interests. 

Vhc degree of succaaa experienced at an institution in acccaplishing each objective is 
depicted by an objective achievement score. 

Each objective achieveaant acore la calculited by tiunming the dogrie of goal ^^^HHl- 
..pnt ffi f all i^oala relative to a particula r object Ivc. !l, 1 



inHe.Sui.'statcnonts a.'e uCiUzed to interpret iho vl,cationar(V.,c,itlon objectives in | 
terms 01 deairiile programmatic concept, relative to Che educational setting and par- 
ticular progr^uns to be Included In the evaluation, j 
h..' proportionate aiaount each goal will contribute to the objective achievcnent v:ort' 
i> J^tvralned by ^ weight factor. 

Weight factors are derived from importance and effort rntings assigned each goal state- 
ment by the individuals vho are administratively respons^e for the conduct of tliv> 
programs to be evaluated. 

The degree to which each goal is fulfilled is dependent upon the sun of the ratln;?^ 
assigned the outconas pe r talnlna to the specific goal. 



^ < lUc data base consists of numerous process and pr'oduct type data elenents— r*ocess d:i:a 
fS/-^! to determine' the existance of desirable program features and product data to assvs- the 
j i-flectiveness of the features. 

^**lach data elenent is in the form of a personal interviev question or a prescribed ob- 
' nervation— both require a "Yes" or "No" answer- 

i ha..h answer rt..oiv ^n a rating based Qpon the evidence avatlable to suPPort the re.sponse. 

' Uata is to b'j collected via on-campus visits by an evaluation a earn utilizing preparea 
data c6llectlon Insicruaents. 

Data aiulyow iu^.cdures Suvc oeen developed which Include computer program*; to utilize 

in determ ininR we lfht factor^ and objective achievement scores . — 



0. 

I the uodul^-ras'devcloped'by proceeding in a syst^-.tic laannT^fTom the objectives to clu] 

VA/l outcoaes ; a progr am is evaluated by proceeding froa the o utcomes to^ttie object ive^.^ j 

f U-.r-.- 2. S *cto-iax and d^scriptt/e suonary of the major components Included in the evaluation 



Evaluative data were collected, via on~campus visits to the seven 
state schools/state hospitals betWien Vu^rch 17, 1975, and April IH, 
1975. The on-campus, visttations, which took. Torm four to six hours 
"to complete at each institution., generally consisted of an introductory 
meeting with the superintendent of the institution, the data collec- 
tion, a visitation team meeting, and a* concluding meeting with the 
superintendent. Following the completion of the ihstituticnal visits, 
the data relative to each institution wore analyzed and individual 
institutional reports, describing the local evaluation findings, were 
written* However, lor this study, where accumulative effect was 
desired, the data were analyzed and the findings reported for the 
institutions as a group. Figure 3 depicts the degree to which the 
four established objectives of vocational education'were being accom- 
plished- in the seven state schools/state hospitals. 

As depicted in Figure 3, Objective 1 of the established voca- 
tional education objectives ; was accorded an achievement rate of 
75 percent. This oT)jective achievement rate partially stemmed from 
the fact that both a published philosophy of vocational education and 
a set of future plans for vocational education existed at^most 
institutions included in this study/ However, the institutional 
degree of commitment to vocational education, as expressed in its 
philosophy, and the degree of detail included in the future plans 
varied from institution to institution* Special funds provided to 
the state schools/state hospitals for the conduct of the pilot pro- 
grams in vocational education for the handicapped were controlled 
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Established Objectives of Vocational Education 

Figure 3. Degree of success experienced at the seven state 
schools/state hospitals in accomplishing estabr 
: lished objectives of vocational education. 



through accountable financial management systems which met the approval 
of most individuals involved. The primary shortcoming revealed reXa^ , 
tive to Objective 1 concerned the appointment and utilization of 
vocational education advisory committees. Advisory committees had not 
been appointed at a number of institutions, and in most institutions 
where advisory coinmittees had been appointed, they were not effectively 
utilized. 

The 83 percent rate of achievement for Objective 2 indicates that 
students enrolled in the various pilot programs were being provided 



compirehensive y^ational education which related education for living 
to education for earning a living* Verification of this relationship 
was revealed through the atlalyses of the goals associated with 
Objective 2 which indicated that at most institutions the general 
education, prof essional and vocational education personnel had:, 
cooperated V. to organize their offerings and services in relation to^ 
the vocational education programs. It was also revealed, and verified 
by former students' employers, that instruction relative to employer- 
employee relations was. being provided and that a sincere efforX was 
being made to assist students in developing positive work attitudes. 
However, consumer education, an aspect of education fot living, was 
not included as an integral part of the total vocational education 
program at l^eral institutions included in this study. 

Objective 3 addressed the concept of making vocational education 
increasingly accessible to those individuals desiring such instruction 
and was interpre^ted in this evaluation model as meaning the identifi- 

cation, appraisal, and fulfillment of the social and educational needs 

♦ 

of the student clientele. The 93 percent achievement rate for 
Objective 3 (Figure 3) is indicative of the success experienced at the 
various state schools/state hospitals in identifying and appraising tlte 
social and educational needs of the student clientele, and in providing 
specialized remedial instruction to students who possessed social and 
educational deficiencies which could have acted as serious barriers to 
successful pr6gram completion or job. placement « At most institutions 
.a form of career guidance was also incorporated into the pilot programs 
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although it was found that occupational information relative to topics 
such as job characteristics, employment conditions, and worker quali- 
fications was'^seldom collected and used as ^ means of assisting students 
in making informed career (decisions, * ' ^ 

.JThe concept of providing quality vocational education programs 
capable of preparing individuals to enter the world of work in 
recognized occupations was addressad by Objective 4, Consequently, 
the degree to which this objective was accomplished was dependent upon 
the degree to which specified operational procedures and ^instructional 
methods were utilized* in. the concjuct of the pilot programs. Although 
the 79 percent achievement rate for Objeictive 4 (Figure 3) indicates 
that the specified operational procedures and instructional methods 
were being ^utilized extensively in the conduct of the pilot programs, 
the degree of utilization varied considerably from institution to 
institution. However, at most institutions it was generally found 
that: 

" n 'ITie^TlinrlTrcrlaim-matrer-^^ ins tructional 

processes as well as the skill development experiences 
^ received by the students reflected the skill and knowledge 
rerquir.ements of the occupations^ for* which instruction was 
being offered, ' c « 

2. Student learning activities were organized in such a manner 
that individualized instruction, when appropriate, was 
available to all students. 

t 

Am 



i* Facilities housing the pilot programs were* quite adequate - 
and the machines and oqulpnuMit therein were commensurate 
with the instructional objectives of the various pilot jJro- 
grams and representative of what would be found in business" 
ands industry, 

^- Procedure^ for program self-evaluation, job placement, and 
former student follow-up were not "^xj^ly developed or 
formalized.* 

Following the conclusion of the pilot program evaluations and the 
issuance-of the individual institutional reports, the visitation team 
members were asked to prepare a written statement describing their 
.perceptions as to the general utility of the evaluation model and its 
related processes, These« int^ividuals, six in total exclusive of this 
' researcher, observed that^^tll^ydata collection instruments were rela- 
V V .tively simple to admifnister and that the results obtained f rom *the 

program' evaluations a^greed with their perceptions of what was actually 
transpiring in the pilot programs. However, it was noted that the 
. _pbtectives, goals, and outcomes contained ^n the model were probably 



more applicable to vocational education programs in public secpndary 
and post-seconiary institutions than to the pilot programs in the 
state ^schools/state hospitals* It was also noted that most students 
experienced difficulty in responding to the interview questions con- 
tained in the ''Vocational Student Questionnaire/* 



/ 
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Conclusions 

Importance and effort ratings assigned^ the 2A goal statements by ' 
the directors of education utilizing 5-point'Likert~type rating scales 
(1 = little, 5 =r great) resulted in (a) an importance rating mean of 
'4,* 00 or greater for 23 goals, with a standard deviation of less than 
1*00 for 21 goals> and (b) an effort rating mean of less than 4,00 fdr 
21 goals with a standard deviation greater than 1.00 for 22 goals. 
Thus the following was Concluded: 

1. The directors of education, at the institutions included in 
this study considered tlie programmatic concepts expressed in 
the^goal statements as desirable functional components of the 

pilot programs in vocational education for the handicapped; 

* ^ "* 

2. All directors of education 'viewed \he programmatjUc concepts 

^ expre'ssed in the goal statements as important. ^ 

3. Generally, obstacles such as cost,* staff resistance, and 

/ 

lack of facilities and .equipment could be or had been over- 
come in an effort to incorporate; t'^e programmatic concepts 
expressed in the goal sta4:ements into the pilot programs • 

4. The degree pf effort required to incorporate the programmatic 
concepts into the pilot programs va^ried from institution to 



institution. 

Objective 1, which addressed the concept of providing vocational 
education to those individuals who could benefit from such instruction, 
was interpreted in ^nis study to mean that in addition to offering 
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skill development classes an LnstJtutionnl ^atmosphere should be created 
which is conducive to the continued growth, development, and further 
refinement of the vocational education programs, Thus/based on the 
various average achievement scores for the goals, associated with 
Objective 1, it was concluded that: 

1, Through the institutional statements of philosophy and 
formalized future plans for vocational education, most state 
schools/state hospitals included in this study were attempting 

^ to create an institutional atmosphere conducive to vocational 

education. ^ 

2. - The institutional financial management methods utilized to 

disburse and account for the special funds provided for the 
pilot programs were conducive to creating an institutional 
V atmosphere favorable to vocational, education. 
3* The limited use of general and specific program advisory 
committees was not conducive to creat;ing an institutional 
~ — - -~ .atmgsphere^£av^^ to vocational education. 

Objective 2 and. its concomitant^^oaT^statSnents-'addressed the 
relationship of vocational and general education. Hence, the average 
achievement scores for the goals associated with Objective 2 are indi- 
cators, of the degree of success experienced at the various state 
schools/state hospitals in relating education for earning a- living to 
education for living. Thus, utilizing the average achievement scores ' 
relative to the goals associated with Objective 2 as a basis, it was 
concluded that-: 



1. At most institutions included in this study, tha general « * 
oduoatlon, prof eBslonal , and vocational education per&onnol 
had cooperated in orj;anl7:lng their services and offerings ^ 
in relation to the pilot programs in vocational education 
for the^handixapped. 

A 

2. The specific program instruction in most iriBtitutipns was 
based on written performance" objectives. However, the 
objectives were not -generally derived from a formal analysis* 
of required occupational competencies. 

3. Instruction relative to employer-employee delations was being 

4 ^ 

provided and efforts were being made to assist students in 
developing positive work attitudes,, 

4. Consumer education was not included as an integral part of 
the total voGaXional education program at mo,st institutlpns. 

Objective 3 addressed the concept of making vocational education 
increasingly accessible to those inciividuals desiring such instruction 
'and was interpreted in this evaluation model as mieaning the identifi- 
cation, appraisal, and' fulfillment of the social a^nd educational needs 
of the student clientele. As was indicated by the 93 percent average ^ 
achievement rate for this objective, a substantial degree of success 
was experienced at the institutions included in this study in Identify- 

t * 

ing, appraising, and fulfilling the social and educational needs of 

the student clientele. Consequently, it was concluded that: 

1. The inatitutional interdisciplinary teams, with thfc aid of 
i * 

the "Behavioral Characteristics Progression!' (BCP) charts, 

■ ' ^ 

\ 



were successful in identifying 'an^l appraising the social an4 
» * ' . / 

educational needs of the student clientele. . * 

2. Specialized remedial instruction was provided students /wb<J • V 
^ * • * ♦ ' 

possessed sdcial or educational deficiencies which cQuld h^vd 

\ ^ ♦ ' ' • 

^'ict'(6d as serious barriers to successful program completion dr 

. » • ■» 
^* job placement. " . 

* • 

3* At most institution^, a form of^pareer guidance serviced was 
incorporated into the pilot programs and students who pos- 
. sessed reconcilable educational deficiencies were not 
proftibited from enrolling in the vocational > programs. 

A. Occupational Information relative to topics such as job 

characteristics, employment conditions, and worker qualifi- 

cations was* generally not collected and used as a Itoeans of 

* • • \' 

assisting students ill making informed career decisions. 

''. ' ^ 
Objective 4 addressed the concept of providing quality vocational 

education prograrAs capable of preparing individuals to enter the world 
# • 

of work-in recognized ■occu<)ations. Consequently, the gqal statements 



associated with Objective 4 essentially pertained to operational pro- 
cedures and instructional methods and the average achievement scores 
for the various gpals represented the degree to which ±he specified 
operational procedures and instructional* methods were utilized in the 
conduct of the pilot prbgrams. Thus, based on the various average^ 
achievement scores for the goals associated with Objective 4, it w 
concluded that: * ' j 



was 



\ 



1. In most pilot programs, student 'learning experiences were 
directly rehiced to tiic^'skni and knowledge requirements of 
iho oocupatioua for which Instruction was being offered. 

2. Facilities housrrig i:he pilpt programs were adequate and the 
machines and equiplneftb contained therein were commensurate^ 
with the instructional objectives of the various ^)ilot pro- 

^gr^ms and representative of what^would be f^ound in business 
and industry, » 

3. In most* pilot programs , >th^ teaching methods, procedures, and 
fnstructional materials utilized in the conduct of the 
t^e^chipg-learning process were commensurate with the educa- 

s*' tipnal ability levels oi the student clienteleT^ ' . 

4; Program sey^revaluetion, job placement, and former student 
follow*^up procedures w^e not fully developed or formalized. 

Based on wTiytten^tatements p^ejPfred* by visitation team members 
and on the researcher's experiences, the following "conclusions were ' 
drawn relative ta.tfie ge.ner^l utility of the evaluation model and Its 
related processes^ *. ' ^ * 

1. The evaluation model is capable of providing factual decision 
making Information to a specific audience. 

2. ^ The evaluatibn model is capable of discriminating between 

those institutions which are highly successful in achieving* 
established objectives of vofcatlonal education and tho&e 
which are less successful. ^ ) 

3 The\ evaluation model data b^e provides sufficient process^ ^ 
and product related d^ta to determine program effectiveness 



as well as describe various processeis which have contributed 

-- ' • • . * ' • - 

to the Xinal outcoitiea. 

«* 

. A. The data analysis ^procedur^es employed in the evaluation model 
provide 'for comparisons; if deemed desirable, on einher a 
program by program or institution by institution basis, 

5. The data collection methods prescribed in the evaluation 
model (Visitations teams, personal interviews, p.rescribed 
observations) are both efficient; and effective, 

6. The progranmatic concepts expressed by the goals included, 
in this "model are more applicable to vocational education 
programs in public secondary and post-secondary i-astitutions 

* * than to the^pilot programs \in the. state schools/state 
hbsi^itals. ^ • . , , 

ff. Mentally handicapped students, as found in the state schools/ 
state hospitals, cannot adequately respond to the intervl^|j^ 
questions contained in the "Vocational Student Questionnaire/ 

RecoiTjnendations 

. ■ • 6 

Based on the conclusions drawix relative to the utility of tHe 

model, it is recomtpended that the evaluation model and Its related 
processes be submitted to further field testing in public secondary or 
post-secondary Institutions after: 

K Adding "ta, deleting from, or changing the goal stat^nients to 
reflect desirable programmatic concepts of public secondary 
. ,or posC-secondary insti'tutions . 
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2. Conducting a study in an attempt to valiH^e thei revised goal 
statements*' 

3. Submitting the data collection instruments to a stringent 
review in an attempt to remove any ambiguities in the phrasing 
of the interview questions. . 

If the evaluation model and its related processes are to be util- 
ized in a setting which includes vocational education programs for the 



\ 



mentally handicapped^ it 1 recommended _ that : 

1. The goal statements be reviewed by a panel of consultants who 

\ i ' • \ ■ 

are knowledgeable in the fields of vocatioiial and special 
\ education and. revised where necessary ^to reflect desired pro- 

grammatic concepts of vocational education programs for the 
mentally handic'kpped. 

2. ' .The "Vocational Student , Questionnaire" be revised to solicit 
action rather than verbal responses from th^ student clientele. 
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APPENDIX A 

Instrument Utilized to Obtain 
Evaluation Statement Ratings 
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VOCATIONAL PROGRAM 
EVALUATIONS 




EVALUATION STATEMENT 
RATINGS 



1J4': 
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Racings of Evaluations ScaCeroents 
Director of Education 

cvaluadon .should be a process which contributes to further development and, 
stabilization of an existing program. To accomplish this mission, those who 
administer the programs, being evaluated must be afforded an opportunity to 

provide input into the evaiuatlon procedure. TKefeforer you/as Director of 

Education, are being asked to rate the following conceptual, expressions in 
terms of their importance to and _cf fort expended in the pilot vocational 
education programs. 



T',- rate the irr^j'^rtanne of the statement j aomider*: "Hod inTi>nr*tant is it 
fjkdt the expressed, concept become a functional part of the pilot programs 
in vocational education?" 



rate th<=r~effcrr'Tzmplt'ed in"the-statemnty-consider^:^ '^Hm,') mioh effort is 

(**rjiuired to overcome obstaclee* that may stand in the- way of implenhanting 
the concept into the pilot vocational education programs?** 

^Consider obstacles such as: 

a) .cost^of implementaiion 
staff resistccnoe 
* c) facilities and equipmeht 

Please indicate your rating of the fallowing statements" by placing a ciy<^le 
around the number that most closely reflects your thinking. 



1:01 An institution offerltig voca- 

tlonal' education should^ posaess 
a statement of philosophy which reflects 
the institution's purpose and objectives' 
relative to meeting (ihe needs of the 
student clienteles. 



Little 
Importance 

1 2, 



Little 
Effort 
1 2 



Great 
Importance 
3 4 5 
_i I I 



Great 
, Effort 
3 4 5 



1:02 The institution should poisess a 
financial plan which la adequate 
to assure the quality and continuity of 
the vocational education programs. i 



Little 
Importance 
1 2 

I L_ 



Little 
Effort 
1 2 



Great 
Importance 
3 4 5 



Great- 
Effort 
3*4 5 
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1:03 The Institution should possess 

formalized future plans for 
vocational education. 



Little 
Importance 

1 2 



Great 
Importance 





Little 
Effort 

1 2 

i 1 


3 


Great 
Effort 
4 5 
• 










1 Th* in« t4 tutlon should utilize 

a general advisory^ committee in ^ 
planning for vocational education -4nd 


tittle 
Importance 

1 2 


3 


Great 
Importance 

4 < 5 


various occupational ailyi^sory committees ^ 
to assist in developing current currlcu- 
lums for each program. * 


Liccie r 
EffortL. 
1 2^ 


3^ ^ 


ijreac 

Effort 
4 5 










2*01 General education DerB6nnel.~~ — 

professional personr^l, and 
vocational education personnel should 


Little 
Importjuice 
1 2 

L — 1 


3 

L 


Great 
.Importance 

4 5- 

1 J 


work together to organize their offer- 
ings and services in relation to the 
vocational education program. 


juicciie 
Effort 
1 2 


3 

» 


Great 

Effort 
4 5 

• • 










2:02 Specific program instruction ^ ^ 

siiould be based on written per- 
formance objectives which were d^irived 


Little 
Importance 

1. 2 
i i — 


3 

1 


Great 
Importance 

4 5^ 


from an analysis of required occupa- 
tional comne tencies 


^ Little 
Effort 
1 2 


- 3 
• 


Great 
' Effort 
4 5 


1 ' . / - 








2:03 There should be a continual 

analysis and subsequent Instructor 
updating of the occupational competencies 


Importance 

1 2 


'3 


Great 
Importance 
'4 5 


thai need to be taught for a specific 
occupation. ^ 


Little 
Effort ~ 
1 2 
» « 


3. 


Great v 
Effort 

4 5 

• ' » 










2:04 Consumer education should be 

Included as an Integral part of 
the total vocational education program. 


Little 
Importance 

1 ' 2 


3 

1 


Great . 
Importance 

4 5 




Little 
Effort 

. 1 2 
1 1 


3 

V 


Great 

Effort 

4 5 

1 1 
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2:0:) hmp lover-employee relations , - _ ^ 

t_ /i i_ t J ^ 1 . ^ Importance Importance 

Hhould be tnught as an Integral xmpu^uau^c AS 

pnrL of the total vocational education j , , ^ i 

program. 



Little Great 
Effort Effort 
1 2 3 A 5- 



::0h Students enrolled in an effective 
— voeac-ibnaLpcogram should show 


Little 
Importance 


3 


Great 
Importance 
A 5 


evidence of change and growth in attitudes. 


1„_^ 2 

{ li— 


» 


t 1. ^ 


- 


Little 
Effort 




ureal. 
Effort 




1 2 


3 


A 5 








* 


\3:01 Students with reconcilable educa- 
tional deficiencies should be 
""liTfCiauraged-to-enroll in /the vocational 


Little 
Importance 
1 2 

1 i 


3 


Great 
Importance 

A 5 
1 1 


program. 


Effort 


~ »~ . 


"Great ~ — 
Effort 




1 ^,2 


3 


A 5 










3:02 Specialized remedial instruction 
.>houId be^^inade available to stu- 


Little 
Importance 
1 2 




Great 
' Importance 
A 5 


dents who have education^ deficiencies 
which act as. serious barriers-^to suc- 












cessful program completion or Job ^ — 
placement. ' . 


jjilttle 
Effort 

— "1— 


3 


or eat - 
Effort 
A 5 


3:03 The potential number of students 
the vocational programs can suc- 
cessfully accommodate should be ^ 


^' Little 
Importance 
1 2 


3 


Great ^ 
Importance 


enrolled » ^ ' 


Little , 




Great 




' Effort 




. Effort 


4 


1 2 
1 1 


3 


A 5 










3:0A All vocational p^rograms within the 
Institution should be offered on 


Little ' 
Importance 




Great * 

Importance 

A 5 
1 > 


a twelve-month basis. ^ 


1 2 
> i 


3 


/ 


• Little 




( 

Great 


* 


Effort 




Effort 




1 . 2 

1.. L- 


3 , 


'A 5 




^ 






♦ 


> 
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3:05 The.lns.tlJ:utlon^8hould offer a 
structured career gurdan^ie pro- 
gram which enrolls all vocational 
students. 



Little 
Ij9portance 

1- 



Great 
Importanc.4! 

Ji 4 5 



Little 
Effort 
1 2 



Great ^ 
Effort 
3 4 5 



4:01 The instructional practices 

carrle'd on In the classroom and 
laboratory should^ provide students 
yith simulated work experiences 
ref lectTve~~of'"what-vould>^be_exp^ecte 
of them In a vage-eamlng situation. 



Little 
Importance 

1 2 



Great 

Importance 

3 4 5 
_j I I 



Little 
EffortL. 
1 2 



Graat 

Effort 



4:02 Major equipment and machine 

acquisitions should be commen- 
surate with th^ program objectives and 
when Installed, the Items should be 
.complete with all safety devices and 
-placad-^on ,a.^p.reventlve maintenance 
and replacement scheduler 



Little 
Importance 
1 %2 



Great 
Importance 

3 4 5 



Little 
Effort 
1 2 

1 L_ 



Great 
Effort 

3 ,4 ' 5 
1 ■ I 



4:03 The vocational programs should 

be housed In adequate facilities 
that are well .managed and free of ^ 
healthy flre^ and safety hazards* 



Little 
Importance 

* 1 2 



L_ 



Great 

Importance 

3 4 5 

-I I L-i 



Little 
Effort 
1 2 

I L_ 



Great 

Effort 
3.4 5 ' 



4:04 The Institution should possess 

a formalized procedure for placing 
both graduates and non-graduates In an 
employment situation that is in or 
directly related to the area of their 
preparation* 



Little 
Importance 

1 2 

I L. 



Great 
Importance 
3 4 5 



Little 
Effort 
1 2 

\ L. 



Gre^t 
Effort 
3 4 5 
_j L 



J 



4:05 Students in the vocational 

programs should be involved in 
cooperative or other out-of-^class 
swork experiances* 



Little 
Importance 
1 2 

<\ L. 



Great ' 

Importance 

3 4 5 
^ ^ i 1 



Little 
, Effort ^ 

' 1 2 

\ L. 



Great 

Effort 
3 4 5 



1:18 



10? 



/*:0'i The voc^^tlonai Instructional 

staff ••jneV'<?rs should be certified 


Lit t le 
Tinportancti 
1 2 


3 


Great 
Importance 
^^' 5 
t i 


in the areas 1^ which they teach and be 


L 

1 1 




» evaluated in terms of their teaching 
\ , , 

- 1 

^ : -4 


Little 
Effort^- - 
1 2 

U 1 


3 

t 


Great 
Efforc 

1 1 








A;07 Vocational instr^jictors should 
use effective teaching metl^odsy 
ucedures » and instructional materials. 


Little 
Importance 
1 2 


3 


~ ^ Great 

Importance 

4 5 
1 1 


in tho conduct of the teaching-learning 
process. 


Little 




Great 




^ Effort > 

1 2 
1 t 


3 


Effort . 

4 5 
i 1 




~ 4r08 -•The.Jnstitution should possess 

a f 0 nnal iz e"d f o H ow- up_p r o^ced ure 
which yields information relative to^ - 


.-_._Little 

Import ance~ 
\ 2 


3 
1 


Great 
Importance 
4 5 
• • 


the adequacy, appropriateness » and 
* effectiveness of the vocational educa- 


Little 




^ Great 


tion .programs. 


. Effort 




Effort 




1 2 

1 1 


3 
1 


4 5 










A:09 An institution offering vocational 
- ejducation should conduct an annual 
y se'lf-evaluatian for the purpose oJf_ - 


^ , Little'' 
Importance 
1 2 


3^ 

»' 


Great 
Importance 
4 5 


identifying strong apdwe^k points as 

a basis for up-grading the programs. ^ 


XittJ.e 




-Great 




Effort 




Effort 




1 2 
t 1 


3 


4 5 
1 1 
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Instruments Utilized to Conduct " the- 
Personal Interviews and Make the 
-J. Prefecribed Observations 
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EVALUATIONS 
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DIRECTOR OF EDUCATION 
QUESTIONNAIRE 
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DIRKCTOU ()F IjnUCATlON (JURSTIONNAI KK 



Personal Interview format 



Institution 
Instrument : 



This instrument is designed to collect data frqm the Director of Education 
relative to the pilot programs in vocational education. Questions con- 
tained in this instrument are of the following two types: 

0 

" 1. Questions directed to the Director of Education. 

2. Questions you, the interviewer, are to answer after making 
V selected observations*. « ' ' 

/" * f « * o 

Different foini^^Wre used to assist you in differentiating between the 
two types 6f^quei^tlon8. The questions directed to the Director of Education 
begin with a verl):*^iind are placed at the left-hand margin, the questions to 
which you are^.^t^Q^jtsspond are written in statement form and are placed in a 
box as shown belot^ . ^ 



To assist you in tormlng your' thoughts relative to the interview, the ques- 
tions are grouped into related areas with |he title of the area printed in 
caps at the left-^hand margin* 

Inatructions : v . • 



I. 

3. 
'4. 



Read the queetionnaire to familiarize yowpaelf with its 'content* 

Follow the format presented in the questionnaire: 

A9k each queation exactly ae presented, , 

Record the ^^Yea" or "No" response, to the question by placing a pheck ' 
fy/^ on the appropriate line. 

Rate the evidence uped to support the answers accdrding to the' ' 
following ^8 cale : 

0 ^ No ' , . ^ ' 

3 = yeSj but evidence to support the answer ia lacking 
7 a y^flj but evidence used to support the answer is questionable ^ 
10 = yes J ihere is strpng evidence to support i^e answer \ 

Recotd a "Yea" or "No" re&ponoo to the observer statements and rate *^ 
the supporting evidence according to the above scale. 

•> 

If a particular question is not applicable to this institution^ place 
the letters N/A on the appropriate "Rate" line. 



' m 



ill 



Yes Np 



Goal 
Rate No. 



thwii inj.M<ulM)a publlbli a clear staUcm'UL 

ol -it/; phn«)sopliy .'incl ohiecLivlis for vocational 

\)ui}(i the ;a;rt'epu»?w ot objj»ctives describe: 

\iH' ciicnLc'le lo be served? 

the inslLCiU ion' s rt'.'sponslbiliLic's? 

thd imians ihrouj'.h which Che objectives arii 
10^ b»: reali , ^ , ' 

li * h' inn ' ? 6 '< ' /V >^ 'ii c fc(?wc>i t - '/ 
h' ' j."^ ^jl hit vi{ ol','***'ii'^^n before 



The statement of ohilosophy and objectives 
j'ivos evidence of commitment* to Vocational 
education on the part of the governing hod^r 
and the administration of the insti*ttition? 

ihc institution's stated vocational education 
jihilosopliy and objectives are api^roprlate nid 
realistic In iiglit* of: 

^ • 

employment needs of people the Insti- / 
tut.lon is expected Lo scrveV 

needs for up-J»raded or up-tlaued 
, occupational skills? 

trends In occupational requir iinents? 

* employment opportunities for the 

institution's students, graduates, ; 
arid early leavers? 

"Yhe objectives ar\^based on an analysis of 
manpower needs and >^ opportunities ^available 
to the clientele the institution, is expected 
to serve? ' . . v 

The objectives are. based or^anvjanalysis of 
the vocatiopai edutatL6n need? of *the 
clientele the insi i tutiofr^ls expected to 
serve? 



1:01 



1:01 
1:01 

1:01 



1:01 



1:01 

1:01 
1:01 

1:01 



1:01 



1:01 
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The object lA/es are stated in such a, wiry that: 



the general public can understand them? 

they give direction to the institution's 
vocational ^education planning and 
activities? 

they enable the general public to 
deternine the extent to which tHe ' 
institution can be expected to serve 
consiunity needs^ *' 



PLANNING * 

Does the institution have a future plan for 
vocatippal^ education? 

* \ , • 

Do the future plans include anticipated 
budgets? , * * 

Are the vocational faculty ^memBers involved 
ln«developing the future plans? *' 

Have priorities been established in view of " 
educational need& and the financial -structure? 

, <* 

ADVISORY COMMITXEES ^ 



1:01 



1:01 



1:01 



1:0? 



1:03 



1:03 



1:03 



{ias a general advisory committee been 
appointed to assist in* planning for voca- 
tional education? ^ 

Does the membersti^p of the general advisory 
committee represent a broad spectrum of 
* groups concerned with vocational education, 
such as: employers » employees,- organized 
labor, former students/ social and economic 
plann'ing agencies » arid the state ejnployment 
service? 

o 

Does the general advisory comodttee meet 
regularly to consider lia'ttera pertinent to " 
effective operation of the' vocational • 
programs? , ^ ' 

Are minutes of the general advisory aoimnittee 
meetings available for examination and refer- 
ence by the institutional community? 

Is there a record of 'the institution's responses 
to the recommendations of the genera), advisory 
committee? 



1:04 



1:04 



1:04 



1:04 
1:04 
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> 




Goal 


<• 




Yes No 


Rate No. 



FINANCIAL MANAGEMENT 

Does a prepared budget exist for each 
.vocational program? 

Does the budget indicate all anticipated 
sources of funds? 

Ooes the budget provide a^.aomplete overview 
ol anticipated expenditures? 

Are the instructional staff members included 
, in the. development of the budget? 

Arc funds budgeted to provide for travel of, '* 
instructors and students in connection with 
instruction? 

Does the accounting system allow you to 
compute the average annual per-stndent cost, 
for instruction? 

Does the financial management system of the 
institution provide *for the accountability 
of all expenditures? * ^ 

SELF-tVALUATION - • , 

Does this institution conduct a continual 
self-evaluation of the vocational programs? 

Is there a steering committee appointed to 
conduct the self-evaluation? 

Are progr,am instructors , administrators/ and 
advisory committee members represented on 
this steering committee? ^ 

Does the steering committee hold periodic 
meetings? ^ 

u 

Docs the steering committee issue a final 
report which identifies major' strengths, 
weaknesses, and plans for improvement? 

^TAFF EVALUATIONS- 

Is a personnel evaluation plan utilized? 

Was the vocational staff involved in estab- 
lishing the personnel evaluation plan? 



1:02 



1:02 



1:02 



1:02 
' 1:02 . 
1:02 
1:02 

A:09 
4:09 

A: 09 
A:09 

. . A:06 
. A:06 
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114 














Yee 


No • 


Rate 


ooax 
No. 




* 


Is mere a Hystfmatic plan for assessing 
vocational staff effectiveness' in terms of 
currlcular needs? 

Do all vocational instructors meet state ^ 
cert if icat ton. requirements for the area in 
which they teach? 










4:06 
4:06 


. * 




^ Arc the voca::ional staff assignments and 
responsibilities clearly defined within the 
instructional program areas? 








4:06 






within the sctucturfe of the total 
institution? - ' 


— 







4:06 




• 


^ 

Is in-*scrvice training an established and 
continuing facet of the vocational programs?^ , 

VOCATIONAL PROGRAMS 








4:06 






Is the range of .Vocational programs offered 
by this institution sufficient to serve the 
full range of abilities represented in the 
student clientele? 


— 







'3:03 






Have the vocational curriculum offerings been 
organized through the joint effort of both 
^ th<^ general education personnel and the 
vocational education personnel? 








2:01 


• 




Are vocational instructional programs also 
' offered during phe summer months? 

Do tlfc summer sd{iool programs meet the 
^ standards of the regular term except where ' 
special provisions apply? ^ * * - 


— 







3:04 












3:04 






Does the school/hospital employ a planned 
prpgram for identifying students with special 
learning deficiencies? 


— 








3:01 






For those students who are identified aS' 

having-specd-adr~rea'rnlng deficiencies, does 










» 




the school/hospital employ a planned program 
for appraising the nature of the probleja? 








3:01 




1 


Are, students with reconcilable educational 
deficiencies encouraged to enroll in the 
, vocational programs? 

^ - 


If* 








3:01 
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Yes No Rate No. 





Do iSio protessloni^l stall raembtMs (counselors, 
psyohoioj^ists, t>tc.) assist ia placing students 
in voi'ut ionai' programs? 



Flw conduct of the actual programs •how 
evi(Je.u-e of planning? 



1:01 



1:03 



PLACEMEN i' h FOLLOW-UP 

po<?s the institution have an organized plan 
for assisting students In obtaining evploy- 
ment in the- occupation for which they were 
prepared? 



4;04 

4:0A 
4:04 
4:04 
4:04 

4:08 
4:08 



Are the placement functions definitely 
assigned and adequately supported with 
sufficient staff and other resources to 
operate effectively? . ^ 



Are the students madef aware of the pla^ 
services'* 



Are the placement services made available to 
all students In all the vocational programs? 

the instructional straff Involved in the 
placement activities? ^ ^ 

* Does^ the institution have an organized plan 
-for conducting a former student follow-up 
srudv? . 

Are the follow-up' functions definitely assigned 
and adequately supported vith sufficient staff 
and other resources so as to be operated 
effectively? 

Is information available on a prograra-by- 
program basis as to: 




Do Jthe fol 



,*-^f— s tu deui falrlure or withdrawal 
Institution? 

ons for stud^ht failure or * 
al?, 

ow-up efforts Include surveys of 
former students* employers? 

Is feedback from employers and other data from 
follow-up *itudlea used for purposes of Institu- 
tional planning. Improving instruction^ and 
modifying* programfe and services? 



DOE-5 



4:08, 
4:08 

4:08. 
4:08 
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' ~ — Goa^l 

Yes No Kate No« 



Wou'td you say that satlnfactory school- 
emp layer relatlO(ia are evidenced by the 
expressed satisfaction ^of both jthe forner 
students and the foner students* 
ewployers? 



4:08 



Are the follov-up activities evaluated in an 
effort to Improve the data collection methods 
and increase the use made of the follow-up 
studies? ' 



4:08 



COMMENTS 



DOE-6 
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VOCATIONAL PROGRAM 
EVAlUATiONS 




COUNSELOR 
QUESTIONNAIRE 



CAREER GUIDANCE COUNSELOR QUESTIONNAIRE 
Personal Interview Fomuit 



Institution; ,/ 

Inktrmsent: 

This Instrument Is designed to collect data from the Pinreer guidance 
counselor relative to the pilot programs in vocational education. Ques- 
tions contained In this Instriinent arejof the following two types: 

1. Questions directed to the career guidance counselor. 

2« Questions you» the Interviewer » vare to answer after making 
selected observations. 

Different formatr were used to assist you in differentiating between the 
two types of questions. The questions directed to the career guidance 
counselor begin with a verb and are placed at the' left-hand margin. The 
quastlons to which you are to respond are written in statement ft>rm and 
^ar€ placed in a box as shown below. 



To assist you in forming. your thoughts relative to the Interview, the 
questions are grouped into related areas with^the title of the area 
printed in caps at the left-hand margin.' 

IftatruQtiom : ^ ^ • ^ , 

U Kead^HJie^ questionnaire to familiarize yourself with its contents. 

2. Follow tfte format presented in the questionnaire* 

'5-. — As k - e a c h -q ues t io n - e x act ly -as-pr es e n t e d: • 



4. Record the "Yes" or "No" response to the quef^^-ion placing U check 

on the appropriate line. ^ , 

5. Rate the evidence used to support the answers according to the 
fotlcwing scale: ^ , 

0 ^ No ^ 

3 - Yes J but evidence to support the answer ts* lacking 
7 = Yes^ but evidence used to support the answer is questionable" 
10 « Yesi there is strong evidence to support the answer 

6. Record a "Yes" or^"No" t^esponse to the observer statements and 
rate the supporting evidence ^according to the above scale* 

7* If a particular qiiestion is not applicable to this institution, place 
the Letters N/A on the appropriate "Rate" line* . 
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Yos No 



Kate ' No. 



Does tjiis Institution utilize a structur«?d pro- 
gram (career guidance program) ,for assisting 
students in making, career choices? 

Does Ihe career guidance program include: 

vocational information services?' 
personal data collection? 
counseling services? ^ 
placement services? 
former student f ollow-^u'^? 



Do yoM regularly collect, evaluate, and use 
occupational information? 

Do you maintain a cumulative recoi^d of each 
vocational ediicacion student? 

. Does the student ^s cumulative record contain 
current data of the following types: 

personal identification data? 
family Information? 
academic record: 
' standardized test results? 
personal-social characteristics? 
accomplishments? 

educational and vocational planst 
entry > withdrawal, an4 follow-up? 

' Is the assembled Information interpreted to: 

the students on a planned basis? 

the student >parent^; or guardian on a 
planned basis? 

^thc student's instructional staff on a 
planned basis? 

Is the counselorrstudent rat:^o appropriate 
for accomplishing the objectives of the 
career guidance program? * ^ 

Do you have a planned program for identifying 
students with special learning deficiencies? 



3:05 



3:05 
.3:05 
3:05 
3:05 
-3:05 



3:05- 

t 

3:05 



•3:05 

3:05 

3:05. 

3:05 

3:05 

3:05 

3:05 

3:05 



3:05 
3:05 
3:05 

3:05 
3:01. 



COU-1 ' 



Yea No 



Goal 
No« 



If studtnti are identified as having special 
learning deficiencies, do you have a planned 
prograa for appraising tht nature of the 
proH leu? ^ 

Oice the -nature of a student 's special learning 
deficiency is appraised, is there a planned 
program for prcyiding individualized remedial 
instruction? 

Do you maintain contact with these students 
who possess special learning deficiencies? 

Pocs the inst ructiOQal i^taff aid you by* 

contributing to the students* cumulative 
records? 

' referring students to you? 

assisting students to gain additional 
vocational information? 

When vocational students leave this" institution 
and seek employment ^ ar9,\they referred to posi- 
tions in which they have a r^easonable .probability 
of being successful? ^ 

Are students re-enrolled\in •other school 
programs when^it is evident they are not ^ 
making satisfactory progress in their present 
program? c 

Kas the administration- established definite 
policies and provided sufficient funds to 
carry on an effective career guidance program? 

Mas a formalized plan been developed for a 
continuous evaluation and up-dating of the 
career guidance program? 



There is evidence that the career guidance 
services are cumulative and developmental?. 
(The services offered the students are 
ba^ed on their needs and actually appear 
to be assisting them in their career 
development.) 



3:01 

3:02 
3:02 



3:05 
3:05 

3:05 



4:04 



3:05 



3:05 



3:05 



3:05 
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VOCATIONAL ^ROGRAM 
EVALUATIONS 




PROGRAM INSTRUCTOR 

QUESTIONNAIRE 

\ ' ft- 




, PROGRAM INSTRUCTOR QUESTIONNAIRE 

Ptrional interview Format 

IM t i tut ion ; . Program : 



Jnatruaant: j " / , 

This inst^uaent ia. designed to collect data from the instructor relative to 
the pilot! prograns in vocational' education. Questions contained in this 
instrunemt are of the follWlng two types:' 

1. Qiiestions directed to the program instructor. 

2« Qtiestiona you, the Interviewer, are to answer after making 
selected c^servaiions. 

Different formats were xised to assist you in differentiating between the two 
types of questions. The questions directed to the program instructor begin V 
Vith a Y^irb and are placed at the left*h|uid margin. The questions to which 
you are 'to respond are written in statement form and are placed in a box as 
shown below. ^ 



To assist you in forming Vour thoughts relative to the interview, the ques- 
tions are grobped into related areas with the title of the area printed in 
caps at the left-hand margin. 

Inatruotiona : 

Z. Head the queatiormaire to familicirize^yonraelf with ita contenta. <> 

2. Follow the format preaented in the queationnaire. 

3. Aak each queation exactly aa preaented^ ' 

4. Record tha ''Yea*' or **No** reaponae to tha queation by placing a check 
jfi/) OA the appropriate line. 

• 5. Ti^e the evidence uaed»to aupport the anewera according to the following 
acale: - . » 

\ 0 ^ No ^ . » , ' -^^ ' 

• 3 = Yea,, but evidence ,to aupport the anawer ia lacking 
^ » Yea, but evid^hce u^ed to aupport the anawer ia queationabte. 
' 10 ^ Jea, there ia atrong evidence to aupport the anawer 

6. Becord a ''Yea" or ^'No^* reaponae to the obaerver atatementa and rate the 
aupporting evidence according to the above acale. ^ 

If a particular queation ia not applicable to thia program, place the , 
tetterla li/A on the appropriates **Ratd** line. , 



i 
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Coal 

Yi's No Rato No. 



S'l'UDKN'l- ENK()l.l.HKNT 



f. 



thefe an ah.scnoe uf^iinrttasonabto: 

course prerequisites that a student must 
meet before being allowed to enroll in 
this program? 

grade level requirements for admission to 
thi^ Instructional program? 

age requirements ^for admission to this 
instructional program? 

achievement levels required of the student 
to remain in this 'instructional program? 

Are all residents who meet the stated require- 
ments enrolled in this program? 

If a resident is unable to meet particular 
admi.asion req^uir ements , is he/she given help 
In remedying the deficiency? 

Has the professional staff (counselor psy-* 
chologist ^ etc.) provided assistance with 
placing students in this program? 

Are students who can benefit; from the Inst rue 
tion placed ixj^ phis program? 

Are the students enrolled in this, program 
capable of succeeding? 

Can students enroll in this program without 
encountering conflicts w4-th other required 
courses? f, 



The potential number of students "this 
program can Nsuccessfully accommodate are 
enrolled? 



Have the students enrolled in this program set 
occupational goals? 

Are the students* goals commensurate with this 
vocational p^rogram? 

Do tihe students show a desire to make the. best 
use of thei'r tine while In the classroom and 
laboratory? 



3:.03 
3:03 
3:03 
3:03 

3:03 

3:03 

2:01 
2:01 
2:01 

3:03 

3:03 

« 

3:05 
3:05 
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Ym Mo — fMtV 



Gotl 

. No, 



V LEAKNING DISABILITIES 

Do you h«v« atudonts in thit progrw who 
hsv« •poclal losming daflclMciea? 

- If you idtntify various •tudanta aa having 
aavara laaming problaaa vhich nay hindaf 
thair auccaaa int^thia prograa* ia thara 
aoMooa to whom thay can ba rtfarrad for 
raaadial inatruction? 

A 

' If you idantify varioua atydanta aa having 
aavara aocial problaaa which aay act aa 
barriara to thair chan£ia^riruccoaafttl~ ~ 
ai^loyaant, ia thara aoMona to who* thay 
can ba rafarrad for aaaiatanca? 

Do you counaal individually with atudents, 
aa neadad) to aid thaa in: 

■aking caraar daciaiona? 

rawving barriara to laaming? 

raaoving barriara to auccaaaful 
aaployvant? 

la atudant attitudinal changa an important 
objactiva of thia program? 

Ia thara avidanca availabla to auggaat that^ 
tha atudant a hava. foraad m' mon poaitiva 
attituda toward achool aftar anrollingi in 
thia prograa? 

ni« cha atudanta in thia prograa ancoiiragad to: 

anavar quaationa or raact to diacuaaiona? 
aak quaationa voluntarily? 
convaraa with adult a? ' 
confida In adult a? 

INSTRUCTIONAL PROGIUH— OUtoTIVES 

' ^ . % 

Do you hava writtan parformanca objactivad' for 
thia prograa which dapict tha conpatanciaa 
raquirad of an individual aaployad in tha 

occupation? 

Did tlia parfonunca <>bjactivaa ariaa out of an 
analyaia of tha occupational conpatanciaa 
' raquirad of an individiial anployad in' tha 
Occupation? 



3:01 



1 



3:02 



'3t02 

3:02 
3:02 

3:02^ 
2:06 

2:06 



2:06 
2:06 
2:06 
2:06 



2:02 



2:02 
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Yc8 No 



Kite 



No'. 



Are ch^ occupational competencies being taught 
baaed on an .analyala of the akll.ls required of 
Individuals currently working In the occuparlow? 

. CAre the students enrolled In this program made 
avart of the performance objectives? *. 

Do your students understand the performance 
objectives? 

' Do you ^possess a plan for t*Ke systematic 
tevlew and revision of the stated* performance 
objactlvea? ^ 
' \ ' \ ' 
Do you posseaa data^ to indicate Student 
prpgress toward the accomplishment of the 
performance objectives? 

♦ 1. * 

00 the administrators fully understand the 

• performance objectlvas of this Ins'tifuctlonal 
program? 



it Is apparrfnt that the students are 
attempting to achieve the performance 
objectives? 



INSTRUCTIONAL PR0GRAM--METH0D0LOGY 



Have you worked with the general e.<iucatlon 
personnel (reading teachers, social counselors, 
psychologists, ^tc.,) in. organizing the, cur- 
riculum ^content for this program? 



Are written lesson plans prepared for each 

unit, of instruction^ > 

Are red or visual materials used wlt^Sach 
unit of Instruction? 

• ' i * 

\ 

.Are such things as field trips and on-site 
visits used as a method of Instruction? 

Are resou^*:* people from the community 'j" 
iitiliied in the instruction process? 

' Is evidence from student ^progress records used 
in planning additional Instruction and'^indi- 
.vlduallzed study? 

Is space and equipment provided for students ' 
to pursue independent" study? ^ 



2:02 
. -2:02 
. ^ 2:02 

2:02 

i 

2:02' 

1:02 

2:02 



2:01 
4:07 
4:07 
4:07 
4:07 

4:07 
4:07 
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" Goal 








Yes . No ' 


Rate No« 



Are the student learning actfUtles or|aDized 
^ in such a manner that individualized instruc- 
tion, when appropriate, is available to all- 
students?*. 

4f 

Is the reading difficulty of written fnstruc- 
tional Mterials keVed to the reading * 
, coap*tence of the students? ^ ' , ' 

Are suppleaentary instructional aids and 
teaching devices used to provide for 
special interests or liearning problens? , 

Are prdjfrcss charts utilized to irecord 
Individual student progress? 

. Does the daily class schedule allow* adequate 
tiae for instruction? . 

^re^ tha^ class schedules arrangedLto accomiDO- 
date activities such as field trips or . 
off-caucus instruction which aay extend 
beytmd the nomlr time allotaent? 



Instructional isaterials are accurate in 
content and' reflect current occupational 
knowledge and practice?' 

There is evidence that a high, degree of * 
coordination exists between the classroom 
activities and jthe* laboratory activities? 

A sufficient quantity and variety of 
equipnent and miteri^ls are available to 
facilitate ^a nulti-media approach in the 

^instrrttctlonal~*proc«ss? 

There is evidence of a satisfactory system 
of checking, servicing, and storing tools 
and materials? 

Appropriate clothing is^wom by the 
instructor and students^when working in 
the laboratory area7 



Is there a planned hoiisekeeping program that 
encourages student participation? 

Are provisions made for the storage of 
materials and the handling^of heavy items? 



/ 
/ 
/ 



4:07 
4:07 



4:07 



4:01 



3:03 



3:03 



4:07 ■ 



4:07 



~4T07 



4:07, 



4:07 



4:03 



4:03 ' 



i 
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yes No 



Goal 
Race No. 



.Have vou^ practiced a fire drill with your 
students? ' f 

Is there an established procedure for obtain- 
ing materials, supplies » or repair parts 
without loss of time to students or yourself? 

Is this instructional program offered on a 
rjegular on-going basis? 

Is this program offered during the aunmer 
months? 

Does this instructional* program provide 
students with out-of-claas laboratory experi- 
ences such as cooperative work-study? 

Are the out*of-class experiences directly 
related to the training provided by the 
inatructioral program? - 



Xre records kept of the student's performance 
on the out-of-clasa experiences? 

Do you include consumer education as an 
integral part of this program? 

Are any of the following units taught: 

"Chopping techniques? ' 
money management (personal budgeting)? 
use of chacks and checking accounts? 
use of credit? 
~ insurance (Social 'Security , health*, life)? 
methods of savings? \ 



The instructional program appears to be 
sufficiently flexible to provide for 
remedial instruction? 

There i^ evidence to. suggest that the 
instruction provided \a comnensurate with 
the students' ability r- ^ maturity levels? 

There is evidence that this instructional 
program providers for individual differences 
of students and at the same time has suf- 
ficient breadth' and depth to challenge all 
students? 



4:03 



4:03' 



3:04 



3;.04 



4:05 



4:05 



4:05 



2:04 



2:04 
2:04 
2:04 
2:04 

-2^04r 
2:04 



3:02 



'3:02 
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Yes No 



Required occupational skills that »re 
current and up-to-date are being taught? 

Required related skills that are current 
and up--to-date are being taught? 



EVALUATION & THE INSTRUCTOR 

Is there a continual self*evaluation of this 

progran:?' 

Did you or someone else from this, prolgram 
have the opportunity to provide input into 
the self-evaluation? 

Have you nade an effort to strengthen 
identified weaknesses? ' ' 



Has your assistance been sought in fonmilating 
current and long»range plans for vocational 
education? 

Are your Inst rue tional ^ssignnents and 
responsibilities clearly defined: 

within this program? 

within the'^^structure of the total ^ * 
institution? 

- Do yoju currently hold a valid teaching 
certificate for this instructional area? 

Has your teaching effectiveness been evaluated 
by those in charge of ttie admin is tratioh of 
this— program?'^ . — ; 



Were you involved In designing the personnel 
evaluation plan? 

Do administrative policies which affect' you 
arrive in written form? 

Have yoii worked in the trade you are now 
teaching (summers or part-time) within the 
last two years? 

Do you regularly participate in technical 
workshops y demonstrations, or short courses 
pertaining to new developments in your field? 



t Coal 
Rate ^ No? 



2:03 



\ 



2:03 



A:09 



c 4:09 



A:09 



1:03 

^:06 
4.: 06 

4:06 

4:06- 

4:06 

4:06 

2:03 

2:03 
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Yes ' No 



Rate 



Goal 
No. 



Do you hold memberships in professional 
organizations relative to your field of 
specialization? 



Do you possess a -'good working relationship/ 
>with the general education personnel? 

Is the general education staff knowledgeable 
of what you are attempting to accomplish in 
this prograjB? 

Do you ppssess a good wotking relationship 
with the professional staff (counselor, 
psychologist, etc.)? 

If the^professional staff (counselor, psy- 
chologist, etc.) knowledgeable of what you are 
attempting to accomplish in this program? 

ADVISORY COMrilTTEE' 

Has an advisory committee been appointed for 
this program? 

0 

Does the membership of this advisory committee 
represent both employers and employees of the' 
occupations for which this program is preparing 
individuals? 



-2:03 



2:01 



2:01 



2:01 



2:01 



1:04 



1:04 



Does the advisory committee -meet on a regular 
basis? . ^ " 

Are minutes of the committee meetings available 
for examination and reference by the institu- 
tional xomraunity?_ - - 



Are Che recommendations of the advisory coranittee 
implemented into this instructional program? 

Do you go to the'advisory committee to seek 
recommendations pertaining to curriculum 
development? 

BUDGET 



"7 



1:04 



1:04 



1:04 



1:04 



Do you possess a budget for this program? 

Does the budget provide sufficient funds to 
•operate this program? 

Have funds been budgeted to take care of daily 
operating costs such as supplies and materials? 



1:02 



1:02 



1:02 
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Goal 

Yes No Rate- No. 



Is there an organised accounting system for 
handling funds that are received or disbursed 
in connection vlth this program? . . 

Does the> budget for this program Include 
travel funds? 

Can you spend funds budgeted for this program , 
''vlthout iBOliy through an excessive amount of 

red cape? 

>• 

MACHINES 4 EQUIPMENT 

Do you have ac your disposal » sufficient 
machlhes and equipment to prepare an individ- 
ual to enter the' occupation for %rhlch he/she 
Is training? 

Is there a replacement schedule to cover all 
major equipmsnt 'and miachines usedUn this 
instructional program? 

Is there a preventative s«inc«nance schedule 
to cover all major equipment and machines used 
in this instructional program? 

Does the final selection of major equipment 
items reflect an agreement between you^ your 
supervisor y ^your advisory committee, and 
your administration? 

Do the students demonstrate a concern for 
vasting consumable instructional supplies? 

PUCEMENT & FOLLOW-UP 

I. , 

Are job placement activities an Integral part 
of this instructional program,? 

Are you involved with placing students in 
employment situations? 

Are you involved with the follow-up activities 
being conducted at^this institution? 

Have you discussed with your students the 
importance of their contribution to future ' 
follow-up studies? ^ 



1:02 



1:02 



- 1:02 



A:02 



4:02 



A:02 



A:02 



2:06 



A:OA 



A:OA 



A:08 



A:08 
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Goal 

Yes No Rate Ho. 



OBSERVATIONS 



iXve to be anawered whi Lc obaerving the 
fheiVitiea and the r»tudent9 at work. 



J 



The learning situation is such that all 
students may at all tines be engaged in 
learning experiences directly related to 
their oecupat{oi)al objectives? 

There is evidence that the instructional 
setting^ permits, and that , students accept » 
realistic industry expectations of pro-, 
ductivity at the level for which they are 
preparing? « 

Students appear to exhibit a genuine pridc^ 
in their vorknanship? 

The instructional sitisation replicates best 
industry practices including cleanliness, 
orderliness, accuracy, speed, work methods, 
and efficiency? 

Skills and knowledge being learned by the • 
students are those curr^ently acceptabli^ in 
the occupation or subject being taught? 

Students practice and exhibit understandings 
of safe work habits? 

All elements of instruction (labprnlory, 
related. and occupational theory) are 

_jdire,ctly-^relatcd^to^requtre»ent8«of— the — 

occupation? i 

All, students enrolled in this program are 
placed in satisfactory work stations? 

There is evidence thar' the following ^tems 
pertaining to employer-employee relations 
are being taught: 

pride in workmanship? 
willingness to work? 
self-discipline? ^ 
responsibility? 

■adjusting to working regulations? 



^:01 



A:01 



A:01 



4:01 



4:01 



4:0L 



4:01 



3:03 



2:05 
2:05 
2:05 
2:05 
2:05 



lNS-9 

er|c . - ' 1^3' 



\ 



> 

• 


import Anctt o T pc rsonsJ progress ? 








2:05 




getting along with fellow ^employees? 








2:05 




Hajor equipment Items are In accordance with 
the Instructional objectives of this prograa? 


— 


V 





A:02 


f 


The major equipment Itepis and machines are 
representative of what Is actually used In 

Industry? 






V 


4 :02 


« 

t 


The major equipment Items appear to be*: 








i 




in good working order? 









4:02 




veil maintained? 








4;t)2 




There Is an absence of unkept and cluttered 
floors and work stations? 





— 




4:03 




^There are adequate provisions for protection 
from electrical hazards?' 





— 


^ 


4:03 




There are adequate provisions for fire 
protection? 








4:03 




The laboratory provides adequate work 
stations for all students .enrolled in the 
program? 








4:03 




It is evldenl that students are aware of 
and are participating in the organization 
and management of the laboratory and 
classroom? 










4:03 


4 


The laboratory and classroom arc arranged 












— for-most*"effectl"ve*njse by^the strudenrsnatfd — ^ 
the instructor? 








^ 


4:03 




The classroom and laboratory are well 
heated* ventilated, cooled, and lighted? 











4:03 




Adequate visual control and supervision of 
all instructional areas is possible? 








,4:03 




An approved system of color dynamics is 
applied to the interior of the laboratory 
and to the machines and equipment therein? 










4:03 




Provisions have been made for appropriate floor 
markings in laboratories and other faciiitjes, 
Identifying alsleways, work stations, potential 
hazards, and traffic patterns? ^ 








4:03 
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n — 

Yes 



No 



Rate 



Goal 
No. 



133 



Ooui 

Yes No Rate No» 



An adequately located, furnished and 
equipped office area is provided for 

-inatructors-^foir^iannlngr^or-^keepinR 

records, and for consultation and, 
adralnlstr./tlon? 

Appropriate tirst aid supplies are 
available and readily acce^ssible in the 
classroom or laboratory area? 



COMMENTS 



4:03 ' 
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VOCATIONAL PROGRAM 
EVALUATIONS 




VOCATIONAL STUDENT 
QUESTIONNAIRE 



\ 




VOCATIONAL STUDENT QUESTIONNAIRE 



-A 



Personal Interview Format 



Institution : i„ Program: \ 

Instrument; ' * " 

This instrument is designed to collect data from the vocational student 
« relatiye to the pilot programs in vocational education «. All questions con- 
j^ained in this instrument are directed to the vocational student. To assist 
you in forming your thoughts relative to the interview, the questions were 

arranged in ,such order that e'ach successive question is related to the 

t 

previous onei Therefore, the order of the questions should lead to a normal 
conversation with the student. 

Inatructiona :' . ^ * * * r 

Z. Read the questionnaire to familiarize youraetf with ita Qontenta. 

2. Follcw the format preaented in the queationnaire. 

3. Ask each question exactly qa preaented: 

4. Record the or '*No** reaponae to the queation by placing a' check' 
(^-}—Qj^--^}i0-^M^j^pjfl-(xte"line^^ _ — ^ - 



5* Rate the evidence uaed tQ aupport the anawera according to the 
following acale: 

0 NO^ 

3 = Yea^ but evidence aupport the nnawer ia lacking 

7 = lea^ but evidence uaed aupport the anawer ia questionable 

10 = Yea^ there ia atrong evidence to support the answer 



\ 
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Yts No 



Gdal 
lUte No. 



Do you like the kind of training you are 
.receiving in this class? 

Did you first learn about this program from 
soMone here at the 'school/hospital? 

Have' you been told or shown what you are 
suppose to le^m in this class? 

Do you want a Job like th^ one in this progran? 

Can you naae some Jobs thAt you think you will 
be^able to do after you complete this program? 

Do you plan tq ; gep ^Job as a when^ 

you leave the school/hospital? 

Do you feel that the training you ^re 
receiving will help you get a Job? 

Do you feel that you are learning how to be 



VAien you finish the program^ will the 
school/hospital help you find a job? 

Do you like school more now after enrolling in 
this class? 

Uhile in tlakn^ are you always busy either 
.working or learning? 

Do you know how to operate all th& tools and 
equipment used in this program? 

Do y>ni like the way the other students in this 
classXtreat you? 

When you\have problems with your work in this ^ 
class, doVother students help you? 

Have you been told about things you will have to 
do when you get oA a Job, such as: 

get to wprkXon time? 

do good Wbrk\ll the time? 

practice saf^ WoriT habits? 

do what your supervisor tells you tq do? 

.be proud of your work? 

get along with othe^ workers? 



3:05 

3:05 

2:02 
3:05« 

3:05 

3:05 

2:03 

2:03 

A:OA 

2:06 

2:06' 

4:02 

2:05 

■4 

2:05 



2:05 
2:05 
2:05 
2:05 
2:05 
2:05 
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^ • E^IPLOYER QUESTIONNAIRE 


« 




\ 

V 




"^^^^^^^^^^^ ^v^^ Telftph6ne Interview Formtt 

"'KmDloVe~r — ' ~ Studentr's Name 














• 


AddreHfl 


Student's Institution , 








Tnl#»phon#» ' Student Program 
















• 


, I. 


h'aad the queQiionmive to yamitiaHMe yourself "with its oontente. 


• 






0 

w « 


Folloto the fomai presented in the questionnaire. 


4 




i 






Ask each questipn exactly as presented.^ 












Record the '7ee" or '^No'\ response to the question by placing a 
check on the appropriate line. 




\ 






Rate the evidence used to support the answers according to the 
following scale: 


• 








0 ^^ No 

3 = Yes, but evidence to support the answer is 


t 

lacking 




4 






y'= Yea, but evidence' used to support the answer is questionable 










10 = Y(t3y th(>vc is strong evidence tcp support the answer 












Yes No Rate 


Goal 
No. 




• • 


When began working for you, did ' 
he/she possess sufficient skills Co do the 
vork expected of him/her? 

* » 


♦ 


2:03 


t 




Is the quality of. work satisfactory? 




4:01 






Does 


produce prpductlve work? . ^ 




4:01 






. Is . able to operate the existing 
ej^uipment? 




4:02 




> 


Does 


seem wl'lllng to learn? 




'2-: 05 

> 


> 




Does 
work? 


\ express an Interest In his/her 




2:05 






Does 


practice safe work habits? i 

V* 




4:01 

\ 
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> 
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\ 
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) 

• 


1 • 

i 











Coall 






Yes >No 


H.ite So.l 



I able to work without :|r\ 



It , 

'excessive amount of supervision, i.e.. 

'"'^^re supcrvisAQn_Jthan As^ruqu^cejl 
of other employees?' 



Does 
Does 



^et to work on timu eacK day? 
have a low absentee rate?^ 



If given tlic opportunity, would you hire 
anotluir Individual who wcjit through the 
same typo of training program? 



A:01 
2:05 
s 2:05 

A:01 




Computer Program: 
' - . \ ^ . Goal VJelght Calculations 

■ , , • ■ . • . • / 
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Goal Weight Calculations 



The computer program described on this page and displayed on the 
succeeding , pages Is utilized to calculate the weight factor for each goal 
statement. This program, written In FORTRAN IV for a WATFIV compiler, 
reads In as data Input the importance and effort ratings assigned f^ach 
goal statement as recorded pn the data collection Instrument entitled 
^'Evaluation Statement Hatlngs." . Printed output for this program includes 
not only the calculated goal weights but also the range, mean, and stan* 
dard deviation for each rating a^ale as well as a title page and a list- 
ing of the data input cards* 



Importance and effort r/atlngs^are punched into the data input cards^ 
beginning in Colijmn 1, exact^ly as they are» found on the data collection 
instrument, '^hus. Column l/« Goal 1:01 Importance rating, Coliffiin,2 « 
Goal 1:01 effoi't rating, Cplumri 3 * Goal 1:0Z importance rating, etc. 

All ratings for a^partlculair .instrupaent are plap.ej^^ on one data card* 

When all ratings pevtalnlng to a particular objective have been punched 
(as noted by the tlrst nun^eral in the^goal statement number) an eight ($) 
is placed in' the very nex^ columxi' to signify that the next group of rat- 
ings pertain to a differ eht objective* Immediately following the ratings 
pertaining to Objective a nine (9) must be punched to signify the end 
of the data on that card! The last data card in the input deck must be 
followed by a blank card/* This program, which is designed to process a 
maximum of 100 cards in/a single run, is not capable ^of handllhg missing 
data* ' • / ' ' . ' ' 

J[!-he^-complete card /deck, consisting of the program deck and the data 
ii^put deck, is organized as follows: 



End Card 
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Y/*END 



Blank Card — >^ 

1^2544241851525242528515352518545254535 4^ 



Data 
Input 
Deck 




645554458545555555584555444485555555554 9^ 
(^25544328^24454535384555535385544545444 9^ 



/5152514385 14252535 18415 15 15 1842525 15342 ^ ' 
/$DATA ^' 



Job- Control 
Cards and 
Program 
Deck 




U2 



//•PASSWO»>'5 
//•WAT»^ IV 
//*FOPMS 

C 
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C 
C 
C 
r 

C 
C 
r 

C 

c 
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VAPIAOLC NAMES AND OESCRIPTIONS 

MATFC 133;. 100) IS THE TI TLC AND THe OIMENSJON OP .THF ^ 
AATtim rOMPHSFD or THE RATINGS ASSIGN«:0 TO TMb VARIOUS 
30AL STATEMENTS. THE ROWS 'J N THE ^ATHIX CONSIST OF THE 
NU^r^ER OF CAPDS. M, WMILF THE COL-UMNS CONSIST OF THF 
NUMnrW OF INDIVIDUAL RATINGS 'felVEN EACH GOAL STATEMENT. 

AVG(2) IS THE MFAN OR AVERAGE FOR FACH COLUMN. IT HAS A 

<;iiH^rptPT nr f^iTH^p op ^a** ggCAtisg rnr AygRAcrs of 



c 
c 

c 
c 
c 
c 
c 
c 
c 
c 



TWO ADJACENT CnCUMNS {IMPORTANCE C EFFORT) MUST BE 
SUMMED TO 0eTE«*«INE THE WEIGHT FACTOR. 

X|« X2» XI. X4, f> X5 ARE COUNTERS WHICH ARC USED TO 
OKTERMINF THE PANGE 0^ THE RATINGS. 

r IS TMT OBJECTIVE INDEX ANO RANGES FROM I TO 4. IT IS 
iNCRKMENTEO WHENEVER' AN •'B^ IS ENCCUNTEPED IN ^THE RATE 
MATRIX. 

dp IS THE GOAL INDEX WHICH COUNTS THE NUMBER OF GOALS 
PEP OBJECTIVE. 



r 
r 
C 

c 
c 
c 
c 

C 

c 
c 
c 

r, 

C 
r 
C 
C 
C 

c 
c 
c 
c 
c 
c 
c 



Wf I IS THE WEIGHT ACTOR THAT IS ASSIGNED AFTER THE 
averages' of two adjacent COLUMNS ARE SUMMED. 

/ IS THE Counter which is incrementeo each time a row of 

• ARE PRINTED. THE FIOST ROW ON ITACH PAGE IS NOT 

COUNTFD. AFTFP,5 ADDITIONAL UlN^S At'E PRINTED. A N«?W 
PAGE OF OUTPUT IS STAPTED. 

17 COUNTS THE NUMQEP 0^ PAGES CF CUT PUT. 

K IS USED AS A SJR5CRIPT ^OR THE VARIABLE AVG AND 
RANSrS BETWEEN AND ••2«'. 

L IS AN IM)EX WHICH C-OUNTS THE NUMBER 0*^ NQN-ZEPP 
EL«^MENTS IN ROW I O^ NTHE RATE MATRIX. THIS ALLOWS FCR 
ONLr NON-ZERO ELEMENTSv, OF THE INPUT DATA TO BE PRINTED. 



M COUNTS THE NOMOEP OF DATA CARDS READ AND THUS^ 8ECCMES 
THF ROW COUNTER IN THE RA^^ MATRIX.' 

5;AVG is T>€ sum 0^^ AVGC-n. "ixfclPOPT ANCE. AND AVG(2)« 
ERFURT. FOR ROUNDING PURPOSES. •OS IS ADDED TO SAVG. 
THIS GIVES A NUMBER ROUNDED Td\THE NEAREST TENTH. 
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1 

2 
3 
4 

c 

6 
7 

10 
U 
12 
13 



K = e 

L = 0 
M = 0 
'J « 1 

s .1 
S ^ CO 
SUM s 0 
SUM? s 



xit )(2. XI. 



^Z. RATC(IC(). IOC) 



.0 
O.C 



14 

l*> 
16 
1 7 
I *) 



c 
c 
c 
c 
c 
c 
c 
c 



•••••««*«*««•«««««**««««•««•« ««««4«4«« «•«*«*««« «t ««««««« 

« ' " • 

• READ JN TMF PAW OATA, MAXIMUM 100 CAPOS AND flC ^ • 

« COLUMNS* ^LAST r»ATA CARD MUST CONTAIN ZEPOS. • 

• * • 

• ««««««*««««««««««««**«•*«*«««•«« «••««••««* * «« «• 



00 ? 1=1. I'^C 

P&AO 101. (PATE(I.J). J^l. ) 
|P ( PATPC I .J ) .£0*0 ) GC TC O 
V r M I 
3 CONTI 

g MMi s H - 1 



\ 



• rttT^pMINf IHr NUMHFP QP N0^-7ER0 ELfMtNTS ACROSS 
THt Cr)LU«NS O*^ PQW 1 JN THf RATE MATRIX. 



»****•••«»*««•«• 



2"' 
21 
9? 



?4 

?s 
^1 

T 1 



I 1 s 1 

00 4 jr 1 , MO 
4 IP ( PATet I I , J> .GT.'JJ L 



PRINT o^2 

OC 2v I - 1 . 

J>C PRINT ?(n 

PRINT 2C1 

PRINT 2C4 

PRINT ?04 

ORINT ;»o;? 

or 4 1 I rl . 

fi\ PR-INT ^C4 
r'RiNT ,i 

PRINT .'>04 

PRImT JC4 

PRINT .^OS 



U f u 

«•••***•••••••#•** •*4««««««««****4* ••**«•«•«**«••*•*•••* 

ORJNT THI? TITLF PAGf^ ' »• 

« 
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144 



.....I.. * 

r • • 

C • PRINT HCAOtNriS AND PA« OATA« > • 

C ' « ' ^ ^ \ « 



c 



16 . PPJNT 500, Zl \ / 

yr zz ar Z2 ♦ 1 / 

^^ pfiiNT -^oi 

«'> > 10 PCcINT ttOt I « AA Te( UJ) f J^tf LI 

4»{ P»!HT SOOt ZZ - - 



*<» PRINT 505 

A 5 OC J»U 80 ^ ^ 

•1 A XI 3 9 

At* X2 s V 

A^ . X3 « 0 

Ay XA 3 fl 

aA X5 X 0 

c 

C ' . A 

C A C)PTL«MINE THf RANCe 0*" H6SP0NSPS TO TM6 RATING • 

C A SCAU« tOCOl'^C ACTION Wmt CONTINUE TMROUCM 00 A 

C ^ A STATtMENT I UNTIL ALL ELEMENTS IN A ""COLUMN ARE^ A 

^ C A <;uMVf n « * - A 

C ^ A A 

C • A * " A 

C A IF AN IS LNCOUNTFRPD, CONTROL PASSfS TO STATE- A 

C • 'MS-NT 2\ ANO THF OBJrCT|VE COUN TEfiVl 5 INCREMEKTfO* A. 

C * . A 

C . . • A 

c • ir A ••<;- IS ENC^tiNTeR^Ot the run i-s ccmpleteo ano a 

C A ANO TMF PROGRAM TERMINATED* • 

<■ - • . . ' • 

C A A««««A A A 

C 

AQ DO 1 I«|« M > ^ 



5D IP IRATE! I f J) «E0«0 ) GO TO <»CD 

51 IF CRATPCI f J)«FO«*<) GO TO 21 

52 IF (PAT«:( t • j)«(--r)« n xi - XI ♦ I 
51 ir CPA TPCI f J)«CO.i») XJ? = X?. ♦ I 
5A IF (PA TEC I f J )«F0« X-* -s XJ ♦ I 
55 IF (PATrfl • J)«L'0«A ) XA ^ X4 f I. 
'.6 * |F (PA Tr<| , J)«r0«*O r XS ♦ 1 

•A* 4* 

' r A ctUNon '^ACH tupmfnt in tmf matrix to a peal numoep 

C A AMD PCTf^HMlN^ TMf SUM OF TACM COLUMN* 

c 

SUM = SUM ♦ <^L0AT(PATE( I f J) ) f ' * ■ 



EACH rLEMENT IN THE COL\«l^ AND SU** THE 
SOUAKES. 



^9 

^9 



XSUM « FLOAT<^ATE< I.Jt)?«2« 
SUM2 M SUM2 ♦ MSUM 



C 

c 



? 60 
61 
6? 
61 
64 



C 
C 
C 

c 
c 
c 
c 
c 
c 
c 
'c 
c 
c 
c 
c 
c 
c 
c 



c 
c 
c 
c 
c 
c 
c 

c 
c 
c 
c 
c 
c 
c 
c 



OCrtRMtNE THF STANDARD DEVlATlDi^ FOR EACH COLUMN, * 

• • 

• Variables used to calculate the standard deviations • 

• APt : ' * "* . . • 



SUM2 » S UW OF Si 

UM OF THE ELBNENT3 3CUARED- 
MMI s M - I 
S^ = VARIANCE 
4 = NUMBER OF ELEMENTS. N 
^ _= STANDARD DEVtAT:ION 




TMF IF STATEMzNT «»OVlOES A MEANS FOR SKIPPING THE • 
STATEMENT S -c SOf^T^SZ) WHEN A COLUMN HAS ZERO • 
VARIANCE^ * 



5. 



YSUM « S{J¥*t?.* 

S2 * (SUM2 - < YS(IM/Mn /Miv| 

IF <S2«LC«0.0) GO TO 5' 

S = SORTC S2) 

K a K ♦ i * 



• ORTb'RMlNE THE ME:AN FOR THE COLUMN. 



AVG( K) s SUM/M 



OfcTfR^^IN^ IF THE IMPORTANCE COLUMN iKxt) OR TMF • 
• EFFORT COLUMN <X»2 ) IS BEING TREATED. • 



66 



IF < K .EO. 2) «0 TO 30 



1 57' 









• 






































- 

• 




* 








• 








** 














- - • 




c 








<■ 








c 




• 




c 


• IF Ks|* PRfNT (fflAL NUMBER (IC*)* 9ANC£ <501>« AND 


4 




r 


• NJMCIHP OF PH'^^OSSL^: • .HCAN* AN*) STANOARD DcVIATION 


• 




( 


• (^05) ^OH THF *mPOHTA^E CCLUMK* 










4 




' t 


- • » 


4 






• IF Kx? « CONTRliL PA*>Nf •» TQ 5TAT»WP;Nf .»C wMpPL TMt 






t. 


• ^WLIGHI TACTOM OTTfPMINFn ANO PM|NTrO < 5( AND 








- • THE PANOr* WFSPONSrS* M£AN« ANT 'iTANOAPO DEVIATION 








• (5tU AMC "RINTFO ri)P THE eFFOPT COLUMM. - 








* • ■ ^ . .. 


• ■. ■ .,„- 




r 


* 








PPJNT 504 ♦ fc, FP' ^ 








PPINT' U*t ' ^ 




h'9 




PPtNT 5;^?* X! « X2* X3* X4* XS* AVr,(K»* s t 
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S z o*c » ^ * 




71 




SUM s 0*0 




72 




SUM2 « o»C ' - 




7 ^ 




CD TO 2 




74 


u 


5AVC > (AVC(l) ♦ AVG(?)) «- .05 




♦ ' 7S# 




IF < ( SAV; aCtivO a') > .AND •( S AVC .LT vZ. S M W4i 1 s- 1 




7fi 




IF ( ( SAVG .CF* 2«5) aANP • ( 5AVC • LT«i3i 5M WF | > ? ■ 




• rr 




IF ( (5AVG •G':. 7 .5 ) •AND.! S AVG .L T .A •<> M WF 1 = 3 




IB 




|f^ C ( $A VG«Gf:.4.9> .ANC,* 1 SAVG.LT.5 ) > W^l « A 




7V 




IF I <SAV';*G»^.«>'.5>.ANd\(SAV0.1.T.0.5M WEI « 5 " 




10 




I^ MSAVC; .GFi^.S > .ANOvlSAVG.LT .7 .5 1 ) WFl ' 6 




*M 




I^ MSAVC .Oft^.S) .ANO.f SAVC.LT.^.Sn WCl^.« 7 




V' . 




IF ( (SAVr>.Gr*H*5> •AN0.(SAVG.LT.'3.Si » W^-I ^ , f\ 




^ 




I«^ ( ( SAVG •^e.*o,5> tAND. ( SaVG*LT,I v*5 n WEI 3 ^ ' 








PPINT *>t'I - 








PP1NT 1 • XI* X?* X3 • X4* X5* AVG(K)« S« Wf't ^ 




•^#» 




PPINT fit'**. 
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• 






• iN("wrH»fjT cnuNTiPs ANO RBsrr vapia'iles bk^^opc 








'ji'tupning contpol to vhe do-st atf'^lnt , 


• 
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"* • . ■ * 


• 




< 
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c 






*\ r 












IT T 0 " * , 








w* I •? /> 


« # 




^ s C»'/ 








<iux = o»r ^ 








SUM?. = o,c 




- • \ M 




Dp s op f I 








I*-. i /•CO. SI CO TO M |( 








go to ? 








V 





I 
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c 

"c " 

c 

c 

c * 



tr PAGE OP OUTPUT IS CC^PLET^Ot CCNTROt PASSFS TO 
STATCMENT 31, AND *> ^'Sw'PAG^ OP OUTPUT IS STARTFO. 



.1..-., 



Q6 
9T 

101 
IC? 
1 0.1 
1C4 



31 



27 s 22 ♦ 1 
PRIN,T •HI 9 ZZ 
PRINT 505 
7 3 0 
,,<U>~T-^a 



21 



900 



RP 3. 1 
CCNTl KUF 
PRI»s|T ogZ 



R ♦ 1 



C 

c 
c 
c 
e 
c 
c 
c 
c 
c 
c 
c 
c 
c 
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\ • 


• 


FORMAT 


ST ATF=Mf-NTS: 


\ ' ' * 


• 


1 '^1 


a DATA CARD FOR««AT 


\ • f' 


• 


MC 


r PRINT — RAW DATA 


\ . * 


• 




- 20S - T ITLt PAGF 
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- PAGE NU*<BFR 
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*>ol 


= RANCeS. t - 5 


\ • 
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*iV2 


= R^SfCNSfcS — IMPORTANCr 
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sri 


« PESF^ONSPS— EFFORT 


I .i 
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b04 


3 GOAL SUMBHRS 




• 




« UI OF ASTER fSKS 
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f. 
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tci 


FOPM^T 


. (^011 
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106 


i 10 


roPMA T 


( 'O' . 


•CAPO 


1C7 ^ 


. 2^C 


«^ORMAT 
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ro«?M*T 
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TAB • 


109 




VlRMAT 
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Tab, 


1 !<» 




FORMAT 
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1 11 




FQPMA T 
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Va8 • 


112 


?C«i 


Fpr^MiiT 
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1 13 




POf?MAT 
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T 1 2 5 , 


1 14 


SCI 


^OftMA T 
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T25f 
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roUMA T 
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TIO • 
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TIO, 
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FOPMAT 
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VOCATIONAL PROGRAM rVAUUATlOKS 
GOAL STATFMENT RATINGS 
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• pagf: • , 

1«. TJO. 



T15 I 



• 3 • 



T45# 



• I MPORTANCC : • . «VI* 
T92» P*>. 2) 

• rFcpc'T:** T?r, t6C. 

TQ;>* rr,.?, TllA. 'WEIGHT 

: • ♦ I? . T J. *. TIC, 'oc 



TMt, •mean:** T<S7, F5»2. 



•mean: • • ror, Fb.2. 

FACTCPU*. T127, 131 
TIC. I?) ' 



no 
120 
121 
122 



OA 1 



FORMAT 
FORMAT 
STOP 
END 



C« 1 • 1 



INPtT DATA :• I 



V/f DATA 



ERIC 



148 







♦ ft * * « ft 


ft * ft 


ft ft 
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^APPENDIX D 

Computer ProgJam; 
Objective Achievement by 
-Institution and- Brograim 
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Objective Achievement by ^ / 

' I Institution and Program / ;^ 1^ j 

The computer program described on this page an^ displayed on the 
succeeding page&j is utilized to calculate an institution objective ^ 
achievement perqentages relative to each vocational^ education program"^ _ 
This program, wi^itten in FORTRAN IV for a WATFIV compiler, reads iri^s 
data input th^ goal weights calculated in the previous program and t^he 
ratings used to jqualify each interview question and prescribed 'observa* 
tion. Printed output- for this ptrogram includes, a title page, the n^e of 
the ins^ltutionJ and the title of the vocational education program; and, 
for e^ch objective, the goal numbers, the go^l weights, the number^ of 
questions pertaining to e^ch goal, the sum of the ratings assigned to 
each goal, and finally, as a percentage, the institution's objective 
achievement rate* . ' / 

The data input deck for this program is composed of the fqllowing^ 
elements arranged in the c^der pretended; ' " ^ ^ \ 

■ A'i ' , ■ / 

1. Weight {factors: Beginning in Column 1, the weight f^actors are 
punched into the data input /card in the exact sequence as calcu- 
lated in the previous program, i.e.. Column 1 ■ weight factor for 
Goal l:Oi> Column 2 « weight: factor for Goal 1:02, /etc. Weight 
factors pertaining to the goals' associated with a particular 
objective are separated byTa blank colimin. 



2. Institution Name: The institution's name is punclied into the 

' . data. card, beginning. iu-Xo/Lumn !• - - ~ <- - - 

' . I f ■ . / 

3. Program Title: The title of the vocational education program 
is punched into the data card beginning in Column 1. 

■ - - i / ■ . 

4. Rating^: The data input cards containing the ratings assigned 
the interview questions and prescribed observat^ions are arranged 
in the j same sequence as the weight factors, i. e. , ^ratings per- 
taining to Goal 1:01, fojllowed by ratings pertkining to Goal 
1:02, ktc. Beginning inl Column 1, these ratings are punched 
riglit-justif led in 2-column fields, with two nines (99) punched 
into tii'e field Immediately following the lasti data entry. Rat- 

'"ings assigned each goal must be placed on separate cards and all 
ratings; assigned a particular goal must be placed on a single 
card; tihus, the maximum number of ratings per goal is 39. 

. Nfttej; pAtaiJC-ela.tlv-e^Q_.aiLy^.ntimber of Instltjutlons-aod-progr^ 



can be processed in a singlA ccxaputer run. ;The institution name 
card, the program title cardi and the goal rating cards arfe 
repeated each time. 



5. End-of-Data Card: A card containing s in at~ least the first 
five col|mins must be placed af^er the last goal rating card in 
the datai input deck. ^ \ 
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The complete card deck, consisting q£ the program deck and the data 
input deck» is organized as follows: 



End Card- 



End-of-Data Card ^i^zZZZZ 

. / ►f'070910030004061099 



Goal Ratj 



fl010000907030309lOl099 



D. Card8< 



■ ^ft'ROGRAM TITLE 




f?INSTITUTION NAME 

Goal Weights »>(7678 88797 8787 6.6^ 

- »>(^//$DATA 
Job Control / J_ 
Cards and / A— 
Program 
Deck 
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VARIAHLE MMCS AND Oe SC» I PT I ONS 



ANS<40> AMSWFPS IS A VeCTOn INTO *MJCM THE RATINGS 

GIVEN To TMC ••YES" ANU -NQ** ANSWCHS TO ThF INTSPVltw 

miersVtCNS apc PLActo ron cqmhutat ions. 

— WFICMTS--IS A V5:CT0« H^TO WHICH TMf, WCTGMT 
FACTOHS ASSIGNED Tf» CACH GOAL ARE PLACED. Thc PLACL- 
MENT OF THE ELEMENTS IN THIS VKCTOR OROVlOeS THE **AJOR 
KE^V FOR TRANSFERRING CONT«OL TO VAPIOUS STATENCNTS IN 
THIS PROGRAM. THE^fr HEIGHT FACTORS *e:^£ COMPUTED IN A 
P^iCVipUS PROGRAM^ 

'SW£U1<) — STOPPO WtlGHT PACTOPS--IS A VfCTCn THAT is 
USED TO TFMPOPAPILV STQRE '^ELCCTEO WEIGHT FACTORS FOR 
»fUTURi: CPI-PWTATIONS AMD POINTING. 

SSUMANCIS >--':iTOMLl> SUM OF THE AMSWFRS--IS A VfC TOP THAT 
IS USEO TO irrMPOKAklLV STOPF. FOP FUTliPE COMPUTATIONS 
ANO PPINTINO* ^HZ Mi*-* OF AN$(40). 

MM(j5i — IS A vector: v*iich tfmpoharily stores, fcp futupe 

COMPUTATIONS ANO t>P,lNTlNCt THC NUMQEP OF ••YES" ANO "NO" 
ANSWERS IN ANS(AO| FOR EACH COAL. 

RR(I5> — IS A VECTOR WHICH TEMpORAPILr STORES THE OBJEC- 
TIVE NUMBER. P. IN EACH EI.CMENT FOR FUTURE PR^INTING. 

iN'sn iBo > --INST HUT ioN«-is a vfctop into which the name 

OP THE -INSTITUTION IS PLACEO. THE NAME OF THE INSTlTli- 
TION IS HfAO INTO A VECTOR AS INTEGERS FOR LATER CHARAC- 
TER ^ANIPULA) IONS. 

P.';0G(y6»--PPUCf;AM-«l^ K VFCTOR INTO WHICH T>C NAMf Of 
THE PROGRAM DEING ANALYZED IS PLACCO. 

suMwei— SUM or tHf weight factops— is the sum of the 

GOAL STATEMEfJT WEIGHTS FtfP AN OBJCCTIvE. 

n--IS THE OBJECTIVE NUMflER ANO RAMCCS FROM I TO 4 FOR 
EACM PROGPAM. 

^^JJ^^N — fCRCCNT-13 THE PERCENT CP OBJECTIVE ACHIEVE- 

c 

L--I5 A CnuNTEK WHICH I G INCPEMENTCU EACH J tfiE A 0*1* 
CARD CONTAINING T^f: RATED "VES" ANO '•NO- ANSWERS IS 
READ. -L'« THEN OECQMES THE NUMBER 0«- GOALS PER ^ 
OBJECT I VE» 

K — is A COUNTER WHICH IS iNCREMLNTeU EACH TIME THE 
VECTOR AN-;(40| IS SUMMED. ^"K«« ThEN BECOMES THC SUB- 
SCRIPT FOP THE OTHER VECTORS THAT TFMPORARILY STORF DATA 
FOP FUTURE COMPUTATIONS AND PRINTING. 
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. 3 



20 
21 

22 

^^ 

2« 



C 

c 
c 
c 
c 
c 
c 



c 
c 

c 
c 
c 
c 
c 
c 



Dl>«FNSION ANSI40)* SSU^AMU^t 

INTFCei M!n5<^)« S«EI(15)» ^^il^U SUMWEU «. PP<15>. prccfN 



• PRINT TI TLF, PACF 







PRINT 900 






^00 5 l« U 7 






5 PPINT ACQ 


T 




PRINT 6CI 


A 




PRINT 60? 






PHI NT 60? 


1*; 




PRINT 6^3 


1 1 




DO 6 1= !• S 


4? 




6 PRINT 6C2 


t Y 




PRINT &C4 


14 




PRINT 60? 


lb 




PUINT 602 


16 




PRINr 6CS 


ir 




PRINT 602 


16 




PRINT 6C2 * 


19 




PRINT 601 




c 





• RCAC THC »EICMT 5 ACTORS THAT WEPE PPEV lOUSL^ 

* ASSIGNED TO TMF COAL STATeMENTSt 
• 

♦ INITIALI^F COUNTFRS ANO VARIABLES* 



READ IOC* (WEKM* I^'IV SOI 
9 R = I . 

J = 0 f 
K = 0 

L = 0 " - 

SUMWEI X 0 



26 
27 
26 



'l*f Al) rUf NAM»r or THf INSTMUTICN AND THf PRO'.PAM 
THAT I", Tn »\r Alv4*LY/rO« THf I «■ T A T f Ml NT 1 1> U",'' 

TI) r>hTrPMiNr m ah oata caphs mavi mi t^e**^. 

TH^ LA<T OATA CAPO •^U'^T COKTAIN 7 f t tl^ \ V* 

F iRST Few C«H.UMnS* 



s\l ( I ) • 1*1. 



READ 101* lIKSt^KD* l«l* ^0) 

ir ( ( INST I ( 1 UEO •XX) .ANO^C lN5iT I ( 2)«eO • XXM CO ^0 09 
READ 10 It (PROC(n« l^t* 80) V 



ERIC 



m PRINT TMF NAME Or'THt INSTITUTION ANO PROGRAM AT 

♦ THE TOP OF A NEW PACE 0^ Ol/fPUT. PRINT 502 IS A' 

• ROW pE ASTERISKS. 



PRINT 500* (INSTMD* i = l* aO> 
PRINT 5CI« (PROC(I),. 801 
PRINT 502 



m ^THr: COAL STATEMfcNTS POP EACH OBJECTIVE WERE 

• SEPARATEO OY A BLANK ON THE FIRST DATA CARD. 

• THFREFORE, THE PttESENCE OF A^^EPO AT THIS POINT 

« wfLL INDICATE THE COMPLETION OF THE ANALYSIS FOR 

« ONE OfJJECTIVE AND CONTROL taILL PASS TO STATEMENT 

« 39 FOR THE SUOSEOUENT PR INT INC OF THE RESULTS. 

• IF NO ZERO IS PRESENT,. A DATA CARO IS READ. 



IQ J = J I 

IF ( WEI ( J >*EO*o> GO rn 3 9 



« 

• READ A DATA CAWf> WHICH CONTAINS THE RATINGS CIVFN 

* THt -YES" AND "NO" ANSWERS TO THE QUESTIONS CON- 

• CEPNING A SOFCIFIC GOAL STATFMENT, "L^ IS INCPE- 

* MENTPO TO INDICATE THAT A C*TA CAPO HAS BEEN REAO. 



PEAO 10?. (ANS(n. I = l. *C) 
U "= L ♦ I 



• SC*N THf VFCTOP. ANSt40)* TC DETERMINE THE NUMRtR 

• OF ans.*fps containeo on the data card. "M" is 

• INCPf MFNTED ONCF FOR FACH ANSWER* 



Mr'. 

00 1 1=1. 4t/ 

ir (ANsd ).Nr»<;«'^) M z M ♦! 

IF ( ANS( I )«E0.Q«O) GO TO ?0 
I CONTINUE 



4 



A60 



c 

c 

c 

c 

c 

c 

c 

c 



• • 

iNCnrMENT K, WHICH IS USED ONLY fOR A SUBSCRIPT • 
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